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Preface 


Tangible cultural heritage digitised in 3D can be a source of new knowledge, including 
with respect to climate-related impact and adaptation. It has also great re-use potential in 
many sectors, including the creative and cultural sectors, but also education and tourism. 
The innovative re-use of digitised cultural heritage can be a significant contribution 
to a European sense of belonging and to European integration. It can also generate 
jobs and growth, in the creative and cultural sectors, but also for tourism in areas that 
are not among the most known or visited sites. By redistributing tourism flows more 
broadly, such processes will also contribute to making the most visited tourist sites more 
sustainable, by relieving some of the pressure. 

The latest developments in the domain of 2D/3D technologies have led to an expan- 
sion and rapid progress with far-reaching impact in numerous applications spanning 
very diverse areas both for highly-skilled professionals — such as architecture and engi- 
neering for digital factories of the future and advanced simulators for flight and surgical 
training — as well as for the general public now using Virtual Realities, Metaverse and 
enjoying cloud computing for 3D computer games. In general, it has turned out that the 
handling of 3D data poses different challenges but also provides growth for new, exciting 
and innovative opportunities compared to more established multimedia. Some 3D tech- 
nologies are already being used successfully also in the cultural heritage (CH) domain, 
especially in the area of data acquisition, modelling, archiving in local repositories and 
harvesting in digital libraries like Europeana (www.europeana.eu) and very soon at the 
UNESCO Dive into Heritage platform. 

The European Commission launched a unique study (VIGIE2020/654) on 3D digi- 
tisation of tangible cultural heritage to enable cultural heritage professionals, insti- 
tutions, content-developers and academics to define and produce high-quality digiti- 
sation standards for tangible heritage (https://digital-strategy.ec.europa.eu/en/library/ 
study-quality-3d-digitisation-tangible-cultural-heritage). 

The UNESCO Chair on Digital Cultural Heritage at the Cyprus University of Tech- 
nology coordinated the consortium of nine key actors from across Europe conducting 
this unique study, which aimed to identify all the relevant elements for the 3D data 
acquisition of tangible cultural heritage, classifying them by degree of complexity and 
purpose or use. It had also covered the specific types of equipment used throughout the 
different stages of the 3D digitisation process, and all the types of relevant data, including 
geometry, colour, texture and materials. In addition, the study, catalogued the technical 
parameters that determine the level of quality of 3D digitisation; existing digital formats, 
standards, benchmarks, methodologies and guidelines for 3D digitisation; and past or 
ongoing 3D digitisation projects and existing 3D models and data sets that can serve as 
benchmarks. 
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3D digitisation has significant potential and long-term value in the area of cul- 
tural heritage. By signing the 2019 Declaration of Cooperation on advancing the digi- 
tisation of cultural heritage in Europe, 28 Member States and Norway have acknowl- 
edged the importance of 3D digitisation technologies for cultural heritage and the urgent 
need to make full use of them. The declaration also endorses a call for common stan- 
dards, methodologies and guidelines for comprehensive, holistic 3D documentation of 
European 3D cultural heritage assets. 

During the implementation of this study, the idea for this volume originated, with the 
intent of gathering 3D research challenges for the digital cultural heritage domain. Con- 
tributions from renowned researchers in this specific area, study partners and supporters 
were selected. 

The aim of this book is to provide an insight into ongoing research and future direc- 
tions in this novel, continuously evolving field, which lies at the intersection of different 
multidisciplinary areas such as digital heritage, engineering, computer science, material 
science, architecture, civil engineering, and archaeology. Overall, in our opinion, the 
chapters in the book reflect the following 3D challenges in the CH domain: 


1. challenges related to the digitisation of CH objects and transforming them to 
digital/virtual twins; 

. the interplay of geometry and semantics for CH; 

. the organization of large 3D databases in CH; 

. handling 3D data in CH over the Internet and using mobile devices; 

. presenting CH content in 3D to the general public; 

. contributing to the research efforts of CH professionals; and 

. reconstruction of CH objects from virtual to real and their 3D production and use. 
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The acquisition of virtual 3D computer models, typically using laser scanning and/or 
photogrammetry technology but also using computer-aided design, raises various chal- 
lenges related to the huge number of objects that should be dealt with, as well as their 
complexity such as size, material, accessibility, etc., as well as the quality of the final 
data sets: 


e The geometric description of a CH object in some computer-tractable form is clearly 
distinguished from the historical description of the build, meaning and purpose of 
a CH object, which has been studied for centuries in areas like architecture. The 
story-telling and the mathematical model have to be suitably interwoven. 

e Once acquired, the large set of digitally available 3D objects must be properly orga- 
nized with the right paradata and metadata to allow activities like exchange and com- 
parison not just within one institution but Europewide, for example through libraries 
such as Europeana or even globally integrated in GIS systems and geo-maps like 
Google Maps. 

e The presentation/display of 3D CH objects for users who are not IT specialists, 
allowing them to use and even manipulate such objects for practical activities, poses 
a lot of technical questions concerning, standards, 3D web browsers, mobile devices, 
etc. 

e CH in 3D content can be presented to the general public in a gripping, immersive 
setting, capturing people’s attention. This necessitates, however, the easy development 
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of animated 3D scenes and 3D authoring tools for interactive experiences, usable by 
specialists for the CH content who should not have to be very familiar with IT issues. 

e 3D technologies should also help CH professionals in their daily research work, for 
example, assisting — literally — in the search for missing pieces in archaeological 
settings. 


New 3D printing technologies allow on the one hand, the detailed reconstruction of 
3D CH objects in previously unknown fidelity in shape and material and on the other 
hand they are finally giving people viable options to turn available virtual CH models into 
tangible real entities. This can support real CH object exhibitions of previously unseen 
objects and assist in the repatriation of some of these replicas back to their origin. 
Naturally some of the above-mentioned challenges are very well-known from other 3D 
application areas but may have a different twist in the CH domain. The different issues 
are obviously interwoven which becomes apparent when studying the chapters of this 
book. 

The contributions in this volume show that substantial activities have been carried out 
in CH-related projects funded by the European Commission. The EU Project Eureka3D 
(https://eureka3d.eu/) is intended to deliver to Europeana more than 4,000 CH objects 
based on the results of this unique study. Moreover, the EU projects ERA CHAIR 
MNEMOSYNE (https://erachair-dch.eu/), TRIQUETRA (https://triquetra-project.eu/), 
ENIGMA (https://eu-enigma.eu/) and ANCHISE (https://www.anchise.eu/) are using 
the results of the study in combination with breakthrough technologies for 3D reverse 
engineering, reconstruction and recovery of memory in cultural heritage. 
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Abstract. This paper is focusing on the exceptional results of the EU Study 
(VIGIE2020/654) to map the parameters, formats, standards, benchmarks, and 
methodologies relating to 3D digitisation of tangible cultural heritage (CH). The 
overall objective of the paper is to further the quality of 3D digitisation process by 
enabling cultural heritage professionals, institutions, content-developers, stake- 
holders, and academics to define and produce high-quality digitisation standards 
for tangible cultural heritage assets. This study identified for the first time in this 
domain, key parameters of the digitisation process, estimated the relative complex- 
ity and how it is linked to technology, its impact on quality and its various factors. 
It will also present standards and formats used for 3D digitisation, including data 
types, data formats and metadata schemas for 3D structures. 


Keywords: Complexity - Quality - 3D Digitisation - Cultural Heritage 


1 Introduction 


This paper summarises the results of the Study on Quality in 3D Digitisation of Tangible 
Cultural Heritage (VIGIE 2020/654), represented in full in the extensive Final Report. 
The work was based on the combined efforts of the in-house study team at Cyprus 
University of Technology (CUT) and a group of nine (9) sub-contracted collaborators, 
together with individual external experts whose research inputs are included in the results. 

The study was organised according to a structure of five (5) main process-oriented 
tasks together with separate project management and dissemination tasks. A mid-term 
workshop was organised to provide expert validation of the interim findings. The Final 
Report combines the conclusions from the nineteen (19) planned Outputs (OU) of the 
study. 


© The Author(s) 2023 
M. Ioannides and P. Patias (Eds.): 3D Research Challenges in Cultural Heritage III, LNCS 13125, pp. 1-33, 2023. 
https://doi.org/10.1007/978-3-03 1-35593-6_1 
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2 The Process of Digitising Movable and Immovable Tangible 
Cultural Heritage 


According to UNESCO, the term Tangible Cultural Heritage (CH) can be classified in 
the following two categories: 


e Movable CH, which ranges from photographs, books, manuscripts and paintings to 
metals, ceramics, glass, wood, leather, textiles, tapes, and other stone variations and 
composites. 

e Immovable CH, which consists of buildings, land, and other historically valuable 
items, typically with fixed foundations connected to the terrain, such as monuments 
and archaeological sites. In addition to castles, houses, mansions, and towers, it 
also includes churches, monasteries, rectories, townhouses and palaces, rural folk 
architecture, technical and industrial monuments, theatres, museums, plague columns 
and shrines. In this category we can also find underwater (shipwrecks, underwater 
ruins and cities) and cave sites. 


Both categories contain objects often heterogeneous (made of different materials by 
using diverse techniques) with an inherently complex geometry and surface texture. For 
example, structures such as monuments with sculptural or pictorial decoration, or one 
of the oldest types of archaeological artefact, the jewelery, have a complex form. 

The digital recording of CH is an essential step in understanding and conserving the 
values of the memory of the past, creating an exact digital record for the future, providing 
a means to educate, skill, and communicating the knowledge and value of the tangible 
objects to the society. Therefore, the primary goal of recording is to know and understand 
the values and significance of the CH object - historical, scientific, aesthetic, social, and 
economic. In addittion, the digital representation of CH objects, structures, and envi- 
ronments is essential for practical analysis, conservation, and interpretation. Selecting 
the ideal technology and workflow for the 3D digitisation of tangible CH objects is a 
complicated, very challenging procedure and one that requires careful consideration. 

However, there is no internationally accepted framework or methodology for spec- 
ifying the quality of detail and accuracy in CH digitisation. Documentation projects 
are typically determined on a case-by-case basis using the many available methods and 
often require significant multi- and interdisciplinary cooperation. An object needs to 
be carefully examined and inspected in order to define the best available digitisation 
options. 

Therefore, the recording of tangible CH requires a thorough understanding of the 
stakeholder requirements, the necessary technical specifications, the existing environ- 
mental conditions, and the intended use of the final 3D model. Selection of the optimal 
human resources and digitisation technology are usually related to the technical spec- 
ifications, size, complexity, material, texture, location, accessibility, IPR and accuracy 
required. For large surface areas, such as monument sites or architectural mapping, a 


' UNESCO: https://bit.ly/30EEdRB. 
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combination of regular aerial and topographic surveys, laser scanning and photogram- 
metric techniques is often used. In addition to the cost of hardware and associated soft- 
ware, a considerable investment in knowledgeable staff and time dedicated to specialised 
training has to be taken into account. 


Accuracy and Precision 

Accuracy refers to how close a measurement is to the true or correct value, whereas 
precision is how close the repeated measurements are to each other. Measurements can 
be both accurate and precise, accurate but not precise, precise but not accurate, or neither. 
A reliable survey instrument is consistent; a valid one is accurate. 


Planning the Process of Digitisation 

The 3D digitisation of tangible CH is an inherently complex multi-stage process. Project 
planning should accurately address and coherently develop a documentation dataset, 
while keeping in mind project constraints including, but not limited to, environmental 
and safety conditions, available equipment, budget, and timescale. 


Documentation Methods 

Measurement methods for geometric recording range from conventional simple topomet- 
ric methods for partially or uncontrolled surveys to elaborated contemporary surveying 
and photogrammetric methods for completely controlled methods. However, there is no 
generally accepted standard for specifying the detail and accuracy requirements for the 
different geometric recordings of tangible objects. Most answers to such challenges are 
primarily based —at the moment- on cost specifications and time limitations defined by 
stakeholders. 


Active and Passive Recording 

For the 3D geometric documentation of movable and immovable assets, the range of 
object sizes could start from few “mm” and goes up to a couple of thousand metres, while 
the number of acquired points should practically have no limit. These documentation 
methods may be grouped in several ways. Firstly, according to those involving light 
recording and those which do not. Secondly, a form of radiating energy is always used 
for gathering geometrical and visual information, therefore a first distinction can be done 
between penetrating and non-penetrating radiation systems. 

In the penetrating category systems based on X-Rays devices allow the capture of 
inaccessible internal structures and surfaces of small objects. Similar X-Rays devices 
are used in areas such as medicine, mechanical engineering, on the airport security and 
on detail investigations by the police. On a larger dimension the use of cosmic rays 
are being experimented for attempting the 3D scanning of the interiors of the Maya 
monuments, or the Egyptian pyramids. For the non-penetrating 3D, the electromagnetic 
energy covers the visible and the InfraRed spectrum. The latter actually may allow a 
little penetration under the illuminated surface depending on the actual wavelength used, 
ranging from fractions of a millimetre for Near InfraRed (NIR), to several millimetres 
for the Far InfraRed (FIR), used in the so-called TeraHertz imaging. However, for 3D 
applications possible little penetrations inside the material are usually neglected, and this 
is the reason why light sources for 3D never go beyond NIR. Within non-penetrating 
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devices a further distinction has to be done between active and passive systems. The 
main distinction between documentation methods is whether they are active or passive. 
Active recording methods use directed radiant energy to mark a point in space, whereas 
passive methods record the reflected radiation from a surface. 


Indoor and Uncontrolled Acquisition 

Indoor image acquisition is more often used for objects or artefacts in museums or collec- 
tions (which are not allowed to be moved), typically of small or medium size, and presents 
several challenges. The size, weight, special illumination conditions, materials, prop- 
erties of the artefacts, and their interior structure directly influence the documentation 
complexity. 

Uncontrolled acquisition is typically used for outdoor scenes or any other environ- 
ment where the conditions (shadows, weather, etc.) are not under complete control. 
Large scale objects such as buildings, excavations, or archaeological sites still with high 
accuracy demands (mm-cm) are classified in this category. 

The report describes other factors and approaches concerning outdoor acquisition 
in greater depth, which require consideration when planning a documentation project. 
These include video frames extraction, object/site recording, and limitations on acces- 
sibility due to skills requirements, weather, ambient conditions, operating hours and 
lighting conditions, site modification and distance optics. 

Moreover, it is essential to consider the state of condition and remedy options. This 
includes geometrical symmetry, together with factors affecting surface reflectivity, such 
as resolution, distance from the sensor to the object/image scale, angle of incidence, 
safety regulations, focal point/spot size, field of view and precision. 


3 Defining Complexity 


The term complexity can be described as “the state or quality of being intricate or 

complicated” or “the state of having many parts and being difficult to understand”. 

Consequently, complexity characterises a system’s behaviour or an object whose com- 

ponents or elements interact in multiple ways and sometimes follow local rules, meaning 

there is no reasonable higher instruction to define the various possible interactions. The 

report explores in greater depth the meanings of complexity and their application in CH. 
Defining complexity is essential because the level of complexity: 


determines, to a high degree, the technology to be used for a documentation project, 
offers the often-missing link between quality and the purpose of use, 

imposes constraints on both the technology and the eventual intended use of the data, 
connects the stakeholder’s requirements, quality, accuracy, expertise during the digi- 
tisation — and completeness if it expresses parameters like object size and random 
requirements. 


Determining Complexity 
Complexity is inherent in all different tangible CH objects and it is a critical consid- 
eration when planning a geometric documentation project. It refers to the geometric, 
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surface/texture, material composition, and scale/application variants. In addittion, a key 
dimension of complexity resides in the stakeholder requirements also, which may include 
the location, state of condition, the set-up of the data acquisition project, the experience 
of the multidisciplinary operators on site, and the fusion of multiple datasets from dif- 
ferent devices and their users/specialists (equipment and data pre-processing) into one 
archive that can be visualised in an easily accessible and searchable way. 

The main challenges in defining a workable end-user definition of complexity 
relate to establishing equipment needs and acquisition methodologies — and in client 
comprehension of additional costs in complex processes and post-processing. 

An online survey aiming at establishing perceptions of the 3D digitisation com- 
munity (944 responses received from 420 survey respondents) identified the following 
parameters as the top three factors for increasing complexity: a) Surface conditions; b) 
Site access; and c) Quality of scanned data. 

Moreover, interviews during our work with 49 key stakeholders and skilled profes- 
sionals in 3D digitisation showed that among the factors most frequently mentioned were 
the importance of the stakeholders’ requirements for 3D digital documentation frame- 
works, object conditions before/during the recording process, location and environmental 
conditions during digitisation, and the levels of expertise of the people involved. 

The challenge is to manage all related activities that run simultaneously during the 
data acquisition phase to produce high-quality results without losing information. Opti- 
mal digitisation technologies are usually related to the desired technical specifications, 
size, state of condition, material, texture, location, accessibility, and required accuracy. 
These considerations are incorporated into the operational schema developed during the 
study. An object’s material or materials adds a further dimension of information and 
complexity. Representing these traits in 3D can pose significant challenges (Fig. 1). 
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Fig. 1. Information example indicating the degree of complexity — detailed breakdown (Com- 
plexity Radar Pie). 


How is Complexity Connected to Technology? 
It is a fact that some data capture technologies and recording methods are more suitable 
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for specific applications than others (i.e., for special object investigations in the lab - 
such as computer tomography). Selection of the data acquisition technology such as hard- 
ware, and software are usually related to the desired stakeholder requirements, technical 
specifications, size, complexity, material, texture, accessibility, and required accuracy. 
Therefore, the report describes mainstream technologies used for the 3D documentation 
of CH tangible assets, in terms of the degree of complexity. 


Impact of Complexity on Quality 

A comprehensive understanding of object complexity is crucial as it has a high impact 
on various aspects of 3D digitisation. The use of the term in this field has remained vague 
with no clear definition, subjective methodology of calculating or apparent connection 
to quality, purpose-of-use, or other imposed restrictions. In other words, there was a gap 
in the collective understanding of the data acquisition project and object complexity as 
a decision support tool. 

However, the complexity of 3D digitisation in CH as a value of its own cannot be a 
matter of subjective estimation. Still, it can be defined after the stakeholder requirements 
are determined, the project specifications are set, the object’s location and environmental 
conditions are known, and the object is defined. Any definition of object complexity 
should have the following characteristics: 


It refers to both 3D data capture and data processing point cloud/modelling, 

It is calculated objectively, 

It is estimated before the data acquisition phase, 

It connects quality, technology, and the purpose of use, 

It provides alerts and limits to recording and processing phases, 

It offers a meaningful tool for planning both the data acquisition and the 3D modelling 
process. 


The process requires a focus on the complexity of the object and the set-up of 
the 3D digitisation project. Well above the object complexity, the digitisation process 
emphasises the complexity of the process itself. 

The study’s online survey analysed the perception of complexity that experts have 
concerning the use of technologies. In their opinion, complexity is related to the degree 
and the kind of information they want to obtain, issues with software and budget, the 
challenges that the surface of a specific object presents, and the location of a monument. 

Any definition of the complexity in 3D digitisation of CH assets should consider the 
following parameters: 


The stakeholder’s requirements, including total budget and time duration, 

The definition of the object and its detailed description, 

The location of the object and the environmental conditions during the time of the 
documentation, 

Multidisciplinary expertise available for the documentation, 

The data acquisition equipment and software are available, 
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e The knowledge for hardware and software to be used and are available for the pre- 
processing of the scanned 2D (images) and 3D data (3D point clouds). 


Parameters of Complexity 

The report illustrates the study’s research into the complexity and the potential issues that 
would affect the complexity of a digitisation process, including stakeholder’s require- 
ments, object, project, team, environment, hardware/software, and pre-processing. Each 
item is subcategorised into five levels (Fig. 1). 


4 Exemplification of Complexity 


During the study, the set of parameters determining levels of complexity and related 
more broadly to quality were considered in the context of 43 cases (25 Immovable and 
18 Movable). Annex 1 of the final report provides outline descriptions of each selected 
case. 

This work has supported the development of a catalogue of data acquisition tech- 
nologies and their output formats. Other contextual taxonomies have been developed 
for Movable and Immovable heritage, based on UNESCO’s World Heritage conventions 
and recommendations. 

When building a taxonomy of complexity for Movable Heritage, we must consider 
each unique element (object-specific) of the question to refine and define the material. 
An essential requirement for the holistic digitisation of an asset refers to collecting 
data regarding these factors and accurately representing them. Key reference examples 
presented in the Final Report include: 

Movable heritage: 


e The Antikythera Mechanism, National Archaeological Museum of Athens, 
e Untitled object: Museum of Contemporary Arts Thessaloniki, Greece, 
e Neolithic Figurine of Dispilio Lake Settlement, Greece. 


Immovable heritage: 


e Asinou Church, Cyprus and World Heritage Site, 
e Cologne Cathedral, Germany and World Heritage Site, 
e Roussanou Monastery, Meteora Complex, Greece. 


5 Parameters that Determine Quality 


As the terms ‘complexity’ and ‘quality’ are used without a precise definition, this presents 
a significant challenge since tangible CH is exceptionally diverse. 

Quality may comprise of different parameters such as the degree of detail, the geomet- 
ric accuracy of the 2D and 3D shape, the spectral, scale and texture, material properties 
and chemical composition, and structural health monitoring status. These parameters 
can be combined in the following categories: a) Geometry; b) Image; c) Material; and 
d) Structural Health Monitoring. 
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Quality parameters refer to different stages of the 3D digitisation process and vary 
depending on the type of tangible CH and the equipment and methodology used. The 
possible purposes or uses of the resulting 3D material also determine different combi- 
nations and levels of those parameters to identify the minimum level of quality that fits 
the definition. 

From the study’s survey responses, the top three parameters of quality categorised 
as the most important by respondents to ensure quality in the digitisation process were: 
a) surface conditions, b) quality of images, and c) environmental conditions. 

Quality is a fundamental component of the 3D digitisation in CH, and it is an essential 
challenge since tangible CH hand- or natural-made structures are remarkably different. 

The possible uses for the resulting 3D material also determine different combina- 
tions and levels of those parameters to achieve the minimum level of quality that fits the 
definition. It is also essential to distinguish the differences between data accuracy (as an 
acceptable margin of error), precision and resolution regarding the geometry. Accuracy 
refers to the closeness of a measured value to a standard or known value. Dimensional 
precision is a measurement of the repeatability, or consistency, of that measurement 
(Fig. 2). Quality parameters may also comprise the degree of detail, the geometric accu- 
racy of the 3D shape, or the fidelity of the capture of colour/texture. The report puts all 
relevant parameters into three main categories: Geometry, Radiometry, Completeness 
and connects these parameters to Complexity, as discussed above. 
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Fig. 2. Information example indicating the degree of quality — detailed breakdown (Quality Radar 
Pie Chart). 


6 Standards and Benchmarks 


One conclusion of the study’s 49 interviews with key professionals in this domain is 
that there are no standards for planning, organising, setting up and implementing a 3D 
data acquisition project. Some experts mentioned the need to distinguish between the 
standards available for the management, administration of projects, safety, health and 
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accessing the object/site for the personnel, the movement of the objects and the standards 
available for the data. 

The report analyses most usually employed formats, along with a full discussion 
of the two focus area formats, terrestrial laser scanning/3D modelling and photogram- 
metry/digital photography. It also elaborates on the distinctions to be made between 
proprietary and open-format data limitations (minimum or maximum), and on judging 
data correctness in the absence of international protocols for data quality assurance. 
An important observation is that formats evolve as users and developers identify and 
incorporate new functionalities. 


7 Identification of Gaps, Additional Formats, Standards, 
Benchmarks, Methodologies, and Guidelines 


There are no guidelines on ways and minimum amounts of data to be collected or the 
quality to be achieved during data acquisition, which entirely depends on the stakeholder 
requirements. There appears to be little common understanding among the international 
multidisciplinary teams regarding what 2D/3D digital data acquisition standards means, 
as well as the obsolescence when new software does not provide backwards compati- 
bility with older file formats. Also, open-source software communities may withdraw 
support for older formats, if these are no longer generally needed by the community. 
Obsolescence can also be accidental: both businesses and open-source communities can 
be led into erroneous practices for different reasons. Digitisation can generate a consid- 
erable amount of original and post-production data. When defining a project, it is crucial 
to understand the stakeholder requirements about the various production file formats 
to avoid inconsistent deliverables and inoperable proprietary data sets. There are hun- 
dreds of different file formats, noting that terrestrial laser scanners, for example, produce 
raw data in a variety of formats. Proprietary formats, such as TIFF or JPG, are seen as 
robust; however, these formats will ultimately be susceptible to upgrade issues and obso- 
lescence. Furthermore, open-source formats can be seen as being neutral, non-reliant on 
business models for their development; however, they can also be seen as vulnerable to 
the susceptibilities of the communities that support them. 

At the novice level, or for those with limited expertise, it cannot be ignored that dif- 
ferent and more basic forms of guidance may be required to promote skills that enable 
widespread 3D digitisation in Europe. Some of the key questions have been beyond 
that and around the conceptual framework needed to address the use cases for digital 
dioramas, including by adding depth to the current 2D images, and by embedding one or 
more canvases within a 3D scene (e.g., multiple paintings or texts, or music/liturgy asso- 
ciated with a cathedral, temple, amphitheatre, or interior of a suitable model). However, 
with growing user and institutional demands, technical developments, and examples of 
advanced research collection and integration of virtual resources (e.g., Sketchfab, Smith- 
sonian3D, 3DHOP, Potree, ScanTheWorld, Clara.io, morphosource.org, exhibit.so, 
hubs.mozilla.com, sayduck.com, Europeana.eu, etc.), there is a pressing and urgent 
need for a technical specification to ensure interoperability and longer term sustain- 
ability. Therefore, the plan for different multi- and interdisciplinary expert groups such 
as the IIIF 3D Technical Specification Group, CEN and ISO Technical Committees are to 
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continue a collaborative approach to clarifying and specifying interoperable frameworks 
for 3D data, including common ways to: 


annotate 3D media of various types into a shared canvas space, with commentary, 
combine 3D media with audio-visual content within a shared space, 

specify the presentation (placement, orientation, and contextualization) of 3D media, 
embed (extend) in 3D the time, material and story dimensions (as a 4th, 5th and 6th 
dimension). 


8 Uncertainty 


Recognising the challenges and lack of consensus on the expression of uncertainty 
in measurement, different organisations worldwide have collaborated with the world’s 
highest authority in metrology, the Comité International des Poids et Mesures (CIPM), to 
develop a more workable definition. Uncertainty concerning the complexity and quality 
in data acquisition is discussed in further detail in the context of the expression of quality 
3D digitisation in the report. 


9 Forecast Impact of Future Technological Advances 


Expected advancements in 2D/3D data acquisition software combined with artificial 
intelligence algorithms in different devices will make 3D digitisation easier, faster, 
more accurate, and more informative. The automatic compilation of different data types 
from various devices and manufacturers, the extraction and recognition of geometri- 
cal features, materials and environmental issues will create new challenges and impose 
greater demands. Development in this area will likely require new competences, spe- 
cialised expertise and training. New standards, regulations and international accepted 
methodologies for data acquisition will be required. 

Moreover, automatic compression and data transfer through 5G, 6G and strong Inter- 
net connections with many gigabytes of bandwidth from the field to the cloud will soon be 
in place to enhance archiving, real time global use and long-term availability and preser- 
vation. Guidelines for the CH domain will be needed on future formats for data, metadata 
and paradata, ensuring interoperability and data longevity. For analytics, blockchain, 
cloud and mobile computing, ontologies, Internet of Things (IoT), aerial and terrestrial 
LiDAR, and machine learning are just a few technologies that have transformed the con- 
struction industry and will undoubtedly impact the CH sector very soon. The increased 
interest in A/V/MX Reality, UAV s, Artificial Intelligence/Machine Learning, cloud and 
mobile computing will enable these new systems to play an indispensable role in the 
management, documentation, modelling, conservation, interpretation and protection of 
CH. Consequently, the development of these systems will have a direct impact on the 
CH industry (i.e., Virtual Museum, Virtual Sites, Smart Cities, 3D- digital libraries, fab- 
rication and eArchiving). The report explores the potential of these technologies in more 
detail alongside that of open data, Heritage Building Information Modelling (HBIM) 
and the digital twin. 
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10 3D Digitisation Process Complexity 


The complexity cannot be estimated subjectively; it can be defined only after all mea- 
surements of the object are conducted, which means that object complexity is not useful 
for 3D digitisation planning and decision-making. Likewise, its neutrality to the intended 
use re renders it impractical for choosing the best technology or setting up the technical 
specifications for 3D digitisation. Regardless of the definition applied, an indication of 
complexity can only occur after the 3D digitisation of a CH object has been completed. 
This includes obtaining all surface detail, texture characteristics and accuracy metrics 
and making a subjective guess at the object complexity. In practice, this would be a 
fruitless exercise. Therefore, it makes sense to reverse this thinking and start from the 
technical specifications which are dictated by the purpose of the 3D digitisation activity 
in question. In accordance with this argumentation, there is a need to shift attention from 
“Object Complexity” to “Model Complexity”. This means that the focus is not on the 
complexity of the actual object (which is connected only to the data capture phase) but 
on the complexity of the produced model, which is connected to the entire process of 
data acquisition and processing. This may look like a conceptual compromise, but the 
alternatives are worse. In effect, one would have to chose between ignoring this factor 
or making subjective guesses (Fig. 3). 


@® Purpose of digitization 
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@ Transportation 
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@ Format, Standards 

@ Security, Safety 

@ Team (experience, qualifications, 
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@ Enviromental conditions 


@® Dimensions 

@ Weight 

@ Permissions/touchable 
© State of condition 


Fig. 3. Moving from Object Complexity to Process Complexity. 


We therefore define process complexity as the degree to which a process is difficult 
to analyse, understand or explain. One way to analyse it is to use a process control-flow 
complexity measure which examines the control-flow of consecutive processes and can 
be applied to data acquisition processes and workflows; then to evaluate the control-flow 
complexity measure to ensure that a high quality of results can be achieved on time. In 
this study, a ‘process’ is defined as a sustained series of events or actions that effects 
change through a series of stages. It resembles an interactive algorithm where elements 
such as the stakeholder requirements, the 3D object properties (such as professional 
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expertise, equipment) interplay with environmental parameters (see) and reorganise, or 
rearrange entities such as activities, decisions, or contexts. 

Therefore, the proposed Data Acquisition Process Management System (DAPMS) 
provides for the first time, a fundamental infrastructure to define and manage different 
processes in the area of 3D digitalisation of tangible CH objects. The proposed approach 
and the steps to be followed are illustrated in the sequence of graphics within: 


e Figure 4 provides an overview of the parameters to be considered, starting with the 
requirements for the 3D object by the owner/stakeholder and moving through aspects 
related to object description, project definition, team characteristics, environment, 
equipment, and pre-processing to the final deliverables 

e Figure 5 outlines the owner’s/stakeholder’s requirements, in terms of: (a) the tangible 
object, (b) the project related stakeholder requirements and (c) the quality of the 
final results to be achieved (minimum requirements needed for a public tender in 3D 
digitisation of tangible objects); for the latter, the requirements are grouped under 
main categories, referring to Geometry (2D, 3D), Image (texture, scale, spectral), 
Materials and Structural Health Monitoring. 

e Figure 6 illustrates the minimum information required for the description of the 3D 
CH tangible asset. 

e Figure 7 presents the main parameters to be considered for defining a 3D CH data 
acquisition project. 

e Figure 8 is about the often-underestimated role of human resources, putting emphasis 
on criteria to assess the level of qualifications and experience acquired through formal 
(professional) or other (amateur/hands-on) training. 

e The environmental conditions to be considered for a 3D data acquisition mission 
are presented in Figure 9 taking into account different possible locations where the 
project can be conducted. 

e As shown in figure Figure 10, the equipment has two broad categories: Software and 
Hardware. For the Software part, one may have to choose among open source, cus- 
tomised, commercial, or combinations of these. For Hardware, a key differentiation 
comes from whether the project is conducted in an indoor or an outdoor environ- 
ment. The parameters/technologies to be considered for an indoor project are shown 
in Figure 10 and those for an outdoor project in Figure 11. 


Figure 12 represents a logical dynamic graph for a 3D digitisation project and sum- 
marises visually the relation of complexity to quality. Figure 13 then shows the Radial Pie 
Chart tool developed by this study to represent the complexity of a digitisation project. 
This tool lies at the heart of our efforts to obtain a concrete measurement of complexity 
in 3D projects that can be used for practical purposes. Outside the direct remit of the 
study, a DAPMS Application (App) has been developed and at the time of writing this 
report is in the final stage of revision and testing in a series of 3D digitisation CH case 
studies (Fig. 14). 


The Radial Charts for Complexity 
Every complexity factor resulting from stakeholder requests including the assigned time 
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Fig. 4. Overview of the VIGIE2020/654 proposed DAPMS 


horizon, total budget availability/priority and overall vision indirectly controls time allot- 
ment and all resources allocation in general, based on the digitisation purpose, desired 
level of detail, location, type, etc. (Fig. 15). 

Digital CH Data Preprocessing requirements tend to vary significantly, depending 
on the scalability associated with the data to be acquired (often recorded from different 
sensors, as part of disparate sources, in different formats etc.). This in turn imposes 
changes in demand for Data Consolidation/Registration (collection, selection, merg- 
ing or integration), Cleaning (missing value imputation, noise control), Transformation 
(normalisation, aggregation/discretisation) and Reduction (decreasing number of vari- 
ables/cases, balancing skewness); each coming with additional hardware and software 
implications (Fig. 16). 

DCH software requirements tend to differ in terms of reliability, operability, 
compatibility, maintainability, quality of results, security etc. 

The demands in computational power, bandwidth, memory, time, and cost are, how- 
ever, always determined in relation to the corresponding hardware constraints. DCH 
data acquisition alone (multi-sensing), can call for considerable variance in hardware 
selection (cameras, scanners, drones etc.). Additionally, demands in storage capacity 
(cache/physical memory, disk drive and partitions, cloud capacity, etc.), processing 
power (operation/network configurations) and representation (monitors, printers etc.) are 
often determined ad hoc. The constraints in computational power, bandwidth, memory, 
time, and cost restrictions are always decided in relation to the corresponding software 
demands (Fig. 17). 

Environmental conditions (controlled or not) that may be perceived as contributing 
factors of complexity are included here. Both long-term (climate) conditions known to 
interfere with 3D data acquisition in general, such as rain, snow, wind, frost, fog, and 
sunshine, as well as physical measurements that become critical in reporting, such as 
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Fig. 5. The Tangible Object as a parameter of complexity 


temperature, humidity, barometric pressure, wind speed/direction, air pollution, etc. are 
taken into consideration (Fig. 18). 

The Team/expert parameter incorporates all complexity factors associated with 
personnel grouping (team formation, communication, interaction and collaboration) 
including HR responsibility and accountability (Fig. 30). This ranges from user qual- 
ifications and corresponding worldwide recognised certification, licenses, and equip- 
ment/infrastructure distribution, to interpersonal coordination together with quality 
assurance implications in the field (Fig. 19). 

The Project parameter includes all complexity factors pertaining to digital CH project 
planning, performance monitoring and management (Fig. 29). This includes setting up an 
integrated management framework to effectively share resources, experience, knowledge 
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Fig. 6. The Stakeholder Object’s description as part of the complexity 


and expertise in pursuit of collective intelligence, subjected to any physical, operational, 
technical or financial constraints and logistics (Fig. 20). 

Complexity factors stemming from object attributes or specifications (Fig. 28), 
including states of conditions, physical, chemical, and functional properties as well as 
dimensions, classifications, permissions for transportation and any other object-specific 
concerns (health and safety, legal, ethical etc.) regulate the digitisation process. An 
increasing requirement of the CH community and corresponding research institutions 
(such as at the of writing this report running H2020 MSCA ITN CHANGES? project) 
relates to the fidelity of colours, ranging from the usual colour calibration within an 
image-based modelling pipeline, to more demanding reflectance measurements such 
as light-material interactions. Such a requirement is a novel complexity gap for the 
3D modelling pipeline, including the visualisation step. At the moment of writing this 
report, there is still no universal consensus on the best format for rich colourimetry 
measurements (Fig. 21). 


The Radial Charts for Quality 
Image quality in DCH is often defined by spectroscopic features achieved via theoretical, 
experimental, and numerical techniques that strive to meet multi-objective photometric 
criteria (spectral regions). These include Absorbance, Transmittance, and Reflectance 
levels mapping to particular source, range, wavelength and frequency configurations 
(Fig. 22). 

Quality in digital CH is often perceived as an indication of potential detail in an 
image, referring to the relative difference in size (or distance) between the image and 
the (radiometric) features represented on the ground. The quality of the calculated scale 
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Fig. 7. The Project as a parameter of Complexity. 


depends on the accuracy of the measured distance, as well as the spatial resolution (pixel 
ratio), affecting color range and (bit) depth (Fig. 23). 

Image quality in DCH digitisation often comes down to realistic 3D visualisations, 
employing sufficiently detailed rendering techniques, to support object representation in 
multi-dimensional space. That is, calculating and adjusting textures based on recorded 
physical material characteristics such as opacity, contrast, and granularity, to a point 
where external structure approximations reflect the desired shape accuracy and color 
depth (Fig. 24). 

Similarly, with 2D attributes, those same quality factors could concern the 3D aspects 
of geometry, for when generating high-resolution point clouds via specialised equipment 
(multi-view cameras, depth sensors, TOF, etc.), often calling for advanced signal pro- 
cessing tools and (semi-automated) modeling practices. In cases of complex background 
or textures, 3D moving objects, and severe occlusions, relative measures might dic- 
tate computationally intensive self-calibration/registration and synchronisation methods 
(Fig. 25). 

A substantial subset of quality factors relating to computational geometry, such 
as accuracy and precision, may coincide with 2D attributes that could be efficiently 
represented on a coordinate plane. Relative measures are often estimated with respect 
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Fig. 8. The Human Resources as a parameter of complexity 


to requirements in point density and corresponding (lack of) completion, with enhanced 
capturing resolution in mind (Fig. 26). 

Another important DCH quality factor is the extent to which the digitisation process 
responds to adverse structural changes, looking for structural reliability and life-cycle 
management. This implies a meticulous condition assessment that goes beyond common 
compositional analyses to appropriately cover states of conservation, connectivity, foun- 
dation strength/integrity and material quality for large-scale built objects, monuments, 
and sites (Fig. 27). 

All quality aspects in answer to the complexity imposed by the characteristics of 
the material(s) involved such as individual strength attributes like yield, fatigue, tensile 
or toughness could be directly or indirectly, individually, or jointly interacting with the 
overall quality of the digitisation process. To mention a few, these include chemical 
composition, moisture, corrosion, carbonation, resistance and porosity. 
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Fig. 9. The Environmental Conditions as a parameter of complexity. 
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Fig. 10. Data Acquisition Techniques (equipment) for indoor and outdoor 2D/3D digitisation as 


a parameter of complexity. 
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Fig. 11. The Location of the tangible object as a parameter of complexity. 
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Fig. 12. Overview diagram illustrating the relation of complexity to quality. 
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Fig. 13. Radar chart depicting the parameters for complexity. 
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Fig. 16. Layers of the Software and Hardware Equipment complexity parameter 
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Fig. 19. Layers of the Project complexity parameter. 
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Fig. 22. Layers of the Scale image quality parameter. 
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Fig. 25. Layers of the 2D geometry quality parameter. 
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Fig. 27. Layers of the Material quality parameter. 
11 Conclusion 


This unique study paid special attention to the fact that the 3D digitisation of movable 
and immovable CH can be an exceptionally complex process with numerous factors 
limiting the object’s eventual quality. Parameters such as budget, time available, location 
and object’s conditions, accuracy and precision are significant in setting the production 
effort and output standard. This issue is crucial for CH as there are no internationally 
recognized standards or guidelines for planning, organising, setting up and implementing 
a 3D data acquisition project. As acquisition technologies and software systems become 
increasingly accessible, with photorealistic renderings now commonplace, it is even 
more crucial to understand the physics behind the hardware, the fundamentals of data 
capture and processing methodologies. 

The complexity of a 2-/3D digitisation project can be determined after factors such 
as the stakeholder requirements, project specifications, personnel qualifications, the type 
and object’s location and corresponding environmental conditions, the equipment to be 
used, the real object and the pre-processing software are defined. These issues will inform 
the level of production effort. The determination of quality can comprise the degree of 
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detail, the precision and resolution of the geometric accuracy of the 3D shape, or the 
fidelity of capturing colour/texture. This EU study -for the first time- demonstrates that 
complexity and quality are fundamental considerations in determining the necessary and 
required effort for a digitisation project, including the needed high grade or value of the 
output. 

Moreover, highly skilled and competent personnel in place, future technological 
advancements, such as the integration of more efficient artificial intelligence algorithms 
for the automatic merge of different point clouds from different sensors, including recog- 
nition and extraction of geometrical features in faster and more accurate sensors, together 
with greater computing power directly linked with high bandwidth internet connections 
such as 5G on cloud computing infrastructures, will surely result in improvements in 
the process of capturing and processing 2—-/3D data. This will make it easier to increase 
the quality of data acquisition results in the fastest possible period, work with larger 
data volumes and bigger 3D models of higher grade and finally to contribute in the 
implementation of the EU DigitalDay2019 declaration. 
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Abstract. Complexity and data quality of a project are strongly linked together. 
The higher the need for better data quality is, the more complex projects become. 
In digitization, various complexity factors have an impact on the quality, such as 
the object properties or environmental factors. However, it also largely depends 
on the employed technology. Digitization instruments are designed for specific 
purposes and applications and have defined specifications and limitations. Some 
devices perform better with one material or another, and can capture invisible 
detail or geometry at large distance. 

The aim of a holistic documentation often implies to take into account signif- 
icant amount of additional data and of different nature. Equipping devices with 
multiple sensors, allows capturing more information and detail at the same time, 
which means a massive increase of data - intensifying the challenge. Thus, it is 
mandatory to be aware of the strengths and limitations to decide which technology, 
or a combination thereof, fits best for each case. 

Smart technological ideas and sensor design can mitigate the complexity. At 
the example of terrestrial 3D laserscanners by Zoller + Fröhlich, this article pro- 
vides insight to manufacturer’s strategies about ensuring maximum survey quality 
and proposing efficient solutions for complexity reduction in survey projects. The 
technological answers range from sensorial adaptions to sophisticated workflow 
approaches and the incremental use of intelligent algorithms to an outlook on the 
necessary focus for the future. The approaches are illustrated with examples from 
emblematic use cases of outstanding tangible Heritage digitization in Europe. 


Keywords: Digitization - Complexity - Quality - 3D Laserscanner 


1 Introduction 


Cultural heritage projects are manifold and each object or site is unique. Hence, each 
digitization project is challenging in its own way. The approaches to document moveable 
and non-movable objects differ and there is no single technology that fits best for all 
documentation projects. The decision about the instruments to use is primarily based on 
the required level of detail. It directly affects the choice of the appropriate deployed cap- 
turing technology. Thus, it also influences the resulting data quality and the complexity 
of the documentation process. The higher the required detail, the more the capturing 
data and the more time it will take, which increases the project costs. 


© The Author(s) 2023 
M. Ioannides and P. Patias (Eds.): 3D Research Challenges in Cultural Heritage III, LNCS 13125, pp. 34-49, 2023. 
https://doi.org/10.1007/978-3-03 1-35593-6_2 


Towards Sustainable Digitization 35 


The example in Fig. 1 shows an excavated mosaic pavement. The digitization costs 
would vary significantly between documenting just the rough structure and color and 
recording each individual tile in detail. 


Fig. 1. Digitization of a mosaic pavement with an example of the detail provided by the point 
cloud (right). House of Eustolios at Kourion (Cyprus). 


The smallest item size of the documentation defines the level of detail to be adapted 
which includes also an indication about the targeted level of data quality. Hence, the 
appropriate 3D survey technology can be chosen. 

In general, there are three different categories for contactless 3D documentation 
technologies: 


e Active systems, such as lidar, sonar or radar, emit a signal, which is reflected by the 
object and returned to the receiver, where the signal is analyzed and converted into 
3D data. In the following, the focus will be on optical methods. 

e Passive instruments, typically cameras, capture the present state of an object without 
emitting a signal. In terms of cameras, pictures of an object are captured and used to 
establish 3D coordinates with photogrammetry principles. 

e Hybrids combine the strategies and project a light pattern onto the surface, which is 
photographed by a camera. The 3D shape is derived from the observed distortions. 


2 What Makes Heritage Documentation so Complex? Key Limiting 
Factors in Surveys with TLS 


2.1 Technological Aspects 


Each technology has its strengths but also limitations and it is crucial that the user is 
aware of these to choose the best solution for each project. No single technology may fit 
perfectly to all needs. Often, the solution to a holistic documentation is a combination. 
This chapter gives an overview on some of the main parameters, which decide about 
data quality with active optical systems. 


Resolution and Distance to the Object. No matter which optical measurement tech- 
nology (active, passive or hybrid systems) the principle is based on a receiving sensor 
for signal analysis and optical components. Fundamentally, digitization is a discrete 
method, which results in a digital approximation of a surface. The resolution is a fixed 
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sampling raster without automatic adaption to the needs of the current object. The higher 
the sampling amount — thus, the resolution - the more accurately its digital twin will rep- 
resent the object. As a rule of thumb, the minimum resolution needs to be at least twice 
as fine as the smallest required detail. Otherwise, the detail may not be readable from 
the spatial survey data, as can be seen in Fig. 2. 


o)}— = 
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Fig. 2. Principle of minimum resolution for sensing small surface features. 


Resolution is defined by an angular increment and changes with range. As depicted 
in Fig. 3, the point spacing increases linearly with the distance from the sensor. For 
instance, a sampling resolution of 5 mm at 10 m distance means a point spacing on a 
frontal surface of 50 mm at 100 m and of 0.5 mm at | m. 
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Fig. 3. Principle of linear increase of the point spacing by distance from the sensor. 


Focal Point. From photography, it is known that the focus needs to be adjusted for to 
capture a sharp image of object across various distances. Usually, with laserscanning 
devices the focal point is static. It is optimized for the entire range of the instrument and 
cannot be adjusted by the user (Fig. 4). 
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Fig. 4. Relation between spot size of the laser beam and point spacing 


It is an important fact to be aware of though, since it is not beneficial to increase the 
resolution of the sampling raster much beyond the actual footprint size of the laser beam 
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on the object. The maximum factor depends on the system used, however, as orientation, 
it should not exceed 1.5x. Therefore, details of high importance should be captured from 
closer setup positions rather than being surveyed across longer distances. However, the 
instrument should not be too close either, as the focus is usually not closely in front of 
the instrument and often noise increases on short ranges as well. 


Angle of Incidence. As shown in Fig. 5, the resolution also changes with the angle 
of incidence of the active optical signal onto the object’s surface. The flatter the angle 
of incidence, the coarser the resolution becomes. It is preferable to have a frontal view 
onto the object - and thus a steep angle - in order to achieve higher resolution of details. 
This might be easy for a wall, but cultural heritage objects — except paintings - are rarely 
flat. It is also worthwhile noting, that with a flat angle of incidence, less light is being 
reflected back to the instrument, and thus noise in the data can increase. 
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Fig. 5. Principle of incidence angle variation of the laser beam 


Field of View. Each optical survey sensor has a defined field of view. All single record- 
ings need alignment in a common reference system in order to describe the 3D shape of 


an object correctly. 
Da 


Fig. 6. Color mapped scan (extract) and example of single imagery tiles from the integrated 
camera sensor with overlaps (right). Example: Panagia Church of Asinou (Cyprus)! . 
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To be able to align all recordings to each other — a process that is referred to as 
registration - the individual parts need to be captured in a way that they share common 


: Courtesy by Cyprus University of Technology. 
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areas with another part (Fig. 6). An ideal overlap area duplicates the amount of recorded 
area by 20-40%, depending on the situation and the technology employed. Thus, the 
complexity increases with a smaller field of view. 

In addition, the accuracy of the registration process influences the data quality. After 
each pairwise alignment of images or scans a residual error remains, which accumulates 
with the amount of alignments and thus, can influence the quality of the result. Hence, 
a larger field of view of the capturing sensor with more geometrical information in one 
shot allows reducing the errors of the registration process. 


Range. As discussed above, the data quality usually decreases with further distance 
to the instrument, especially regarding the resolution and focal point. In addition, over 
longer ranges, less emitted light returns to the instrument, which increases the range 
noise (error in depth) in the data in an almost linear relation, similar to angular errors. 
While decision makers often focus on the maximum range an instrument can cover, the 
importance of the minimum range is often underestimated. In narrow and tiny spaces, 
such as corridors and technical rooms, a larger minimum range can leave larger blank 
areas in the data, see Fig. 7. These holes require filling with data from additional setups. 


Fig. 7. Data loss (magenta) by min. Range at 30 cm (left) and 60 cm (right) inside a tight space. 


Accuracies. Each instrument is subject to certain accuracies, which manufacturers of 
professional equipment should specify in an appropriate data sheet. High-level devices 
allow verifying at any time in their life cycle, whether the accuracies stated in the data 
sheet are valid and offer the possibility of calibration by the manufacturer, or alternative 
countermeasures. 


Measuring Time. Another fundamental factor for data quality is the measuring time, 
or data rate as it strongly affects the range noise in the data. 


For passive optical sensors, such as cameras, mainly the exposure time and sensor 
sensitivity determines how dark and noisy the image is. 

For active laser scanners, the data rate (amount of measurements per second) has a 
similar impact. A higher rate leaves less measuring time per pixel, which increasingly 
causes more range noise. Therefore, a lower data rate leads to less range noise and 
subsequently to higher quality of the data (Fig. 8). 

In this context, it is important to be aware that the resolution setting of laser scanners 
entail a higher data rate setting in the background, which will increase the range noise, 
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typically by factor 1.4x. Only a few instruments allow adjusting the data rate via a quality 
setting to compensate that effect. However, this of course implies a longer scanning time. 
Therefore, with respect to an optimum of accuracy, data volume and scanning time, the 
resolution should not exceed the real needs. 


Fig. 8. Noise at data rates: fast (left) - slow (right). Panagia Church of Asinou (Cyprus)!. 


2.2 Challenging Object Properties 


The complexity and data quality depend also on the object properties. 


Shape. The shape of the object is a key factor for the setup configuration. Comparing toa 
flat shape captured frontally from very few positions, a staggered shape with concavities 
and convexities requires multiple setups from different perspective and vantage points 
in order to cover the shaded sides. Each scan or photo contributes and provides data to 
fill in gap-areas, shadowed from previous viewpoints (Fig. 9). 


Fig. 9. Shadow effect on the survey data by staggered surfaces (tympanum and jamb figures at 
Cologne Cathedral, west portal) and with filled gaps”. 


2 Courtesy by 3DOM: Fresenius Univ. Koln, Heriot-Watt Univ. Edinburgh, Dombauh. Köln. 
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Depending on the accessibility and the movability of the object, as well as the limi- 
tations of the used technology, some cavities might not be documented at all or require 
the usage of different technology. 


Dimensions. The physical dimensions of the object have a direct impact on the com- 
plexity due to the need of more scan data, like additional small pieces in a big puzzle 
(Fig. 10). The position and orientation of all these tiles need to be established and each 
one needs to share sufficient data with others in order to fit together. Each alignment 
introduces a residual error, which then may accumulate over the entire project. 


Fig. 10. Cologne Cathedral: point cloud of the north elevation with the urban context. Scanner 
setup positions on the roof (right). Geometrical identities due to large-scale symmetries require a 
sophisticated project structure for the correct assembling of scans.2 


Whilst the multiplication of capturing positions at ground level mostly faces orga- 
nizational questions, the multiplication on vertical levels quickly leads to physical 
limitations and cutbacks in coverage and data quality. 


.? 


Fig. 11. Alignment of three 3D scans. Result (left) and single scans (right). 


Symmetry. Especially on a large scale - geometric symmetry challenges algorithms 
and users, when trying to align the pieces together (see Fig. 11) and the likelihood of 
a false alignment increases. Thus, more time is needed for the alignment and makes a 
thorough check of the data even more important. 


Material. The data quality largely depends on object materials. Thus, it is important 
to be aware of its properties and effects on technology when choosing the appropriate 
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digitization system. In this context, the reflectivity of an object is the most crucial factor. 
The more reflective, the more light returns to the instrument, and within limits, the data 
will be less noisy. On the other side, less light is returned by dark or distant surfaces, 
which leads to an increase in noise. 


Further, the surface finishing is important, whether matt or glossy. The latter can 
cause strong reflections, which may even blind the sensor. This effect is well visible on 
polished metal. Fully reflecting materials, such as mirrors, lead to range measurements 
of reflected objects falsely allocated on the vector of the sensing laser beam. 


Fig. 12. Laser light diffusion on marble [7] 


Laser measurements on translucent objects, such as marble, often suffer of higher 
range noise and incorrect measurements, as the light penetrates the surface to a certain 
extent and is reflected from various depth levels of the object (Fig. 12). Translucent 
materials, such as glass, lead to wrong range measurements, as the refraction properties 
of the material are unknown and cannot automatically be compensated for. In addition, 
the state of aggregation matters. Water, rain or fog might have further limiting effects on 
the data capture. Especially water or liquids in general cannot be digitized with common 
technology and may cause artefacts. 


Object Conditions. When looking at the material it is important to look also at the 
condition of the object surface. Vegetation, such as moss, or vandalism may also affect 
the quality of the survey data (Fig. 13). 


we 


Srtreaec et 


D 
Ys ve" 


Fig. 13. Factors of survey data alteration: vandalism and surface flora at Cologne Cathedral”. 
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2.3 Challenging Circumstances and Environment 


Accessibility. The ideal vantage points for the survey are sometimes out of reach or 
may lead to challenging logistics (Fig. 14). Often, the practical aspects of the survey 
equipment - such as weight, size and autonomy — become key factors for overcoming 
these limitations. The accessibility is directly linked to the physical properties of the 
object and its direct surroundings. Vertical extensions and narrow passages lead to steep 
angles of incidence by the sensor signal while digitizing the surfaces. 


Fig. 14. Exposed logistics on the exterior of a tall gothic spire (Cologne Cathedral)”. 


Ambient Conditions: Moving Objects, Illumination. Moving Objects potentially 
cause complications regarding the processing steps of the survey data (alignment, color- 
ing) and temporarily obstruct the sensor’s view towards the object, which results in more 
or less large data gaps. Especially on touristic sites, it can become a relevant limitation. 
However, often these sites cannot be closed to a large extend to public visitors during 
the survey. Then, other strategies become necessary, for example collecting additional 
data from similar vantage points, which may increase the project time and complexity. 
In addition, animals may obstruct the view, especially when present in large numbers, 
such as pigeons (Fig. 15). 


Fig. 15. Hostile climatic conditions and wildlife presence during survey of historic British bases 


in Antarctica’. 


3 Courtesy by British Antarctic Survey and the United Kingdom Antarctic Heritage Trust. 
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Adverse environmental conditions can be hostile and limit the access to the site 
or part thereof, such as temperatures, radioactivity or explosive dust and gases, or just 
weather factors, such as wind, fog, rain, snow, or ice. 

Finally, the illumination has a significant impact on the data quality. While high-end 
laser scanners are not limited by sun or ambient light, it is still critical for additional 
sensors of the system, such as RGB-photography (shadows, color temperature etc.). For 
instance, light conditions may vary during the fieldwork time and induce color variations 
in the final data. 


3 Technological Strategies for Managing Complexity and Data 
Quality in High Profile Projects 


3.1 Software and Workflow Strategies 


The strategies for managing complex projects largely depend on the employed tech- 
nology. A general survey of the topic would be beyond the scope of this paper. In the 
following, the authors will focus on strategies for 3D laserscanner devices and data. 

Complex projects usually imply more data volume to be managed. In terms of 3D 
laser scans, the data needs to be puzzled together with sufficient overlap. The alignment 
process of the 3D puzzle tiles is commonly known as registration. For this purpose, 
dedicated field solutions have appeared on the market, which automatically download 
the scanner’s data and register all scans in parallel to scanning [3, 4]. It allows verifying 
the completeness and data quality of the project immediately in the field that no data 
is missing and sufficient detail is captured. A field software helps managing the project 
complexity by providing tools: 


To automatically align all data and detecting issues during field phase 
To ensure the completeness of data; 

To confirm the level of detail; 

To check the quality of data; 

To plan remotely the data capturing phase. 


Field Registration. Similar to solving a puzzle, the alignment of scans is not a trivial 
task even for powerful computers. In order to simplify the registration process, additional 
markers or physical objects can be placed in the scene to be recorded by the laserscanner 
and located in the data by software. Although markers, i.e. targets, are largely considered 
highly accurate for the alignment of survey data, in the reality of cultural heritage survey 
the usage of such targets is often restricted. In this context, the physical size of targets 
is relevant and in some cases, the placement may cause damage to the surface, e.g. by 
using adhesive tape to attach these to walls or objects. Also, the use of survey targets at 
large scale requires an additional planning step in order to achieve an ideal distribution. 
Thus, it is a significant time factor. 


The so-called Cloud-to-Cloud registration is an established alternative to substitute 
or integrate with targets. The Iterative Closest Point (ICP) algorithm [2, 5] identifies 
closest point neighbors between scans and minimizes the distance between these. It 
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can be considered a statistical fine-tuning of the alignment and delivers reliable results 
especially when the scans are already pre-orientated, typically within meters. 

Another method is to identify certain features in the 3D scans. The so-called Plane- 
to-Plane registration [8] offers an alternative target-less approach by detecting abstracted 
planes that allows initial alignment and an accurate reliable registration. 

Even though automatic processes exist, a registration process is not fail proof and 
often requires manual intervention for completion. In complex projects, the manual 
alignment can be cumbersome and time-intensive. The scan alignment methods without 
targets require 30-50% overlap between scans and it is difficult for an operator to keep 
track of all covered surfaces. Thus, depending on the size and complexity of a site, 
surveyors often tend to scan more than necessary for avoiding registration issues back in 
the office. Addressing these issues, Z + F presented a field registration workflow in 2015 
[4], which, has been widely accepted as a standard principle for 3D laserscanning by 
today. The position and orientation of the single scans is estimated by additional sensor 
information, such as video, inertial systems (IMU) or GNSS. The system combines 
this information with target-less ICP registration on site. In this manner, the operator 
monitors the registration quality continuously and the redundancy is lowered towards a 
minimum level without risk. 

The current systems are often subject to technical limitations and thus, for example, 
are used only outdoors (GNSS based) or indoors (videometry based). Others allow a 
combination thereof. Zoller + Fröhlich proposes the so-called “Blue Workflow” [4]: the 
scan positioning system (a combination of GNSS, compass, barometer and IMU sen- 
sors), which tracks the displacement of the device between consecutive scan positions, 
works in- and outdoors. The software Z + F LaserControl® Scout runs in parallel on a 
WiFi connected tablet PC, automatically downloading and registering (cloud-to-cloud) 
the incoming pre-orientated data. Feedback about sufficient overlap and the registration 
outcome is provided immediately. The registration can also be combined with targets, if 
required. Hence, the operator proceeds with a minimum number of scans without inad- 
equate high redundancy of covered area or potential registration issues. Since the tablet 
PC allows parallel processing, the operator efficiently uses idle time until the completion 
of the next scan. The algorithm may not be able to register all scans automatically due to 
limited overlap, symmetry or noise by moving objects. Therefore, the solution integrates 
a toolset for immediate manual adjustment. In addition, the software allows to examine 
and to assess the data directly on site. The user is able to verify and ensure the required 
point coverage on the focused objects, as well as the data quality of the measured points. 

Such on-site processing tools allow the user to decrease the complexity of a project 
and ensure the best data quality possible, as well as the overall project success. 


Annotations, Metadata and Project Planning. Annotation capabilities of the 
employed software are of great benefit, as starting with the first visit on site by the 
decision-making staff, these tools can log field notes and meta data (as text, photogra- 
phy, audio, video or any format) to create detailed tasks for the field team and include 
referenced photographs about key setups [6]. Further, these tools allow planning scan 
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positions in advance in order to prevent unexpected and significant deviations of the 
main project objectives, which result in increasing costs. 


Data Management. Opposed to industrial applications - where scan data typically is 
replaced with best-fit primitives - every data point may matter for documenting cul- 
tural heritage where irregular, non-planar surfaces prevail. Surfaces of ancient structures 
hardly fit to geometric primitives without significant loss of information. Filling larger 
gaps in the data is not trivial but always work intensive. Therefore, it is good practice to 
multiply scan positions and capture with lower resolution from various different angles 
than to scan with higher resolution from less positions. The coverage increases and an 
overall high point density is achieved by overlapping multiple scans. Generally, depend- 
ing on the complexity and the required data quality, the aim to achieve a maximum 
coverage tends to increase the size of the dataset and the complexity for the registration. 
Further, difficult scan positions require an immediate visual validation option of the 
registration result before moving to the next position. 


The software Z + F LaserControl® works on a scan-by-scan logic for processing or 
analysis [4]. Thus, the solution is scalable to process extensive projects on a tablet PC. 


3.2 Hardware Approaches 


Today there are laserscanner hardware solutions available, which combine the spatial 
data with data from additional sensors in order to mitigate the complexity for the use of 
different documentation technologies. These data are complementary and the processing 
is highly automated. 


Handheld Integration. A static 3D laserscanner is typically able to capture panoramic 
full-dome scans, repeated from various vantage points in order to reach a high coverage. 
At a certain ratio between redundant and uncovered surfaces, the efficiency of additional 
static full-dome scans may decrease. Then the complimentary use of handheld scanning 
devices may be a method to quickly fill-in small remaining gaps. 


Fig. 16. The Z + F scanner can serve as a data hub for handheld devices (here: DotProduct) 
sending the data to the field software Z + F LaserControl® Scout for registration [3] 
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The handheld data integration requires sufficient overlap with the panoramic scans. A 
dedicated workflow from Z + F synchronizes via WiFi F between both devices (Fig. 16). 
The data of the handheld scanner is directly registered and verified its completeness. 


Infrared Camera. Infrared cameras allow reading the surface temperature. For conser- 
vational analysis, it can provide indirect information about the material condition, such 
as thermal bridges, structural discontinuities, electrical faults or voids. In some cases, 
the infrared data can even unveil information from below the object’s surface. 


Combining spatial and infrared data manually requires the identification of common 
points (homologies) in both data sets. This had been a quite challenging task before the 
advent of automatic solutions due to the typically great difference in resolution between 
both systems. 


Fig. 17. Mapped point cloud with infrared image unveiling hidden technical installations. Add-on 
IR-camera on a 3D laserscanner (right) [3] 


Zoller + Frohlich developed a dedicated calibrated infrared system, which senses 
thermal information and spatially allocates each value automatically (Fig. 17). Hence, 
thermal analysis is carried out directly on the 3D geometry, as opposed to 2D images 


[3]. 


High Dynamic Range (HDR) Technology. In cultural heritage applications, the inter- 
est beyond the pure geometry includes capturing high quality RGB information in order 
to map scan data with photographic information. The combination with image data 
is used for surface analysis of materials and assessment of degradation [3]. Further, 
museum and immersive experiences require photorealistic color of the data. 


In this regard, it is challenging to ensure a constant color quality as the lighting con- 
ditions change in the course of a project. In addition, photos can differ in appearance, 
depending on where the internal sensors focused on. The image can result under- or 
overexposed, if the focus or light measuring area differs within the same image frame. 
HDR photography is a standard technique for panoramic photography. Outdoors, the 
scene is usually well illuminated but shaded objects are often too dark. Standard pho- 
tography can hardly find one suitable exposure setting for all areas in an image without 
over- or underexposed parts. Instead of just one image at one exposure setting, HDR 
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merges a set of images with different exposure times into one single image (Fig. 18). 
Thus, this technology is often integrated in laserscanning devices. 


SmartLight. Color scanning in dark environments is a challenge. Due to massive pres- 
ence of visitors at daytime, some applications require to execute the capturing work at 
night hours. Then, illumination of the environments most likely is required, which can 
be a challenge due to cables, shadows and even multi-shadows different light sources. 
Therefore, Z + F sensors operate with a HDR enabled camera system and provide inte- 
grated spotlights to illuminate the scene [3, 4]. These lights are mounted radially around 
the camera optics allowing shadow-free imagery even in near proximity, which is then 
mapped onto the point cloud. 


Photogrammetry. Although deriving from different technologies, the output of active 
(e.g. laserscanners) and passive (e.g. separate, non-integrated cameras) systems can be 
successfully combined. For this purpose, the camera photos are used to create a 3D 
model with algorithms of photogrammetry, using the laserscanning data as a reference 
or skeleton model. Photos from an external sensor are usually taken at the same instance 
with similar lighting conditions and ensure best RGB color results. On the opposite site, 
laserscans provide highly accurate geometric detail. It should be noted that highly accu- 
rate data can be acquired locally on close-range with photogrammetry alone. However, 
for large sites it is much more complex to achieve the same accuracies as with laser- 
scanners. Combining both technologies, the advantages of a combination ensure highest 
data quality in terms of color and geometry [1]. 


Special Equipment. When the physical size of the object exceeds common dimensions, 
then the vantage points for data capturing need to be chosen also in elevation. 


Special fixtures and support structures may be necessary to reach these positions. 
There are different solutions available from technology suppliers, such as lever arms or 
inverted tripods (Fig. 19). The latter can be used to orient a scanner upright or upside 
down. Especially when lowered down to privileged positions, such as below the key 
stones of tall cathedral vaults, it allows to enrich the data significantly at a high quality 
level. 
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Fig. 19. Shaft tripod with laserscanner lowered through a vault keystone and special support 
bracket for scans exterior to the spire (right) at Cologne Cathedral”. 


4 Conclusions and Outlook 


As shown above, technology is key for an efficient documentation of cultural heritage 
and can minimize a project’s complexity while ensuring highest data quality. Also, 
technological supplements or sensor fusion, as shown above, can significantly improve 
data quality. 

Although different solutions are available, 3D digitization is still a time intensive 
process. Thus, there is a trend towards much faster mobile systems, which acquire 3D 
data while the sensors are moved through the scene. However, the systems cannot yet 
provide the same data quality as of static panoramic laserscans. However, mobile systems 
will play a significant role in the future. They also collect a much higher data volume to 
be processed in less time in combination with higher resolution. Thus, it is mandatory 
to explore new processing and storage possibilities. Computer clouds can help in this 
regards, but a dedicated cultural heritage solution is still missing. 

Apart from data capturing, there is still intensive research and development necessary 
to segment and classify the data automatically. At present, the data is visualized as a 
large collection of 3D points or surfaces consisting of triangles. However, the elements 
do not provide information about the object they belong to. Manual augmentation of this 
information is a tedious process and thus automation, maybe with the help of artificial 
intelligence, is certainly one of the future key research topics. 
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Abstract. IIF (International Image Interoperability Framework) is a collection 
of open web standards, broadly used for presenting and annotating high resolu- 
tion images and audiovisual content, adopted by hundreds of GLAM (galleries, 
libraries, archives, museums) organisations worldwide, and permit the reuse and 
recombination of digital assets across traditional digital institutional boundaries. 
While these standards have been successful for 2D images and AV (Audio/Video) 
data, there is a growing interest in how they might be applied to 3D content. An 
example of this would be to combine shards of pottery from disparate collections 
into a single whole within a viewing experience. The IIF 3D Community Group 
has been collaboratively considering common challenges and potential solutions 
to key areas, with major 3D developers and researchers, and ongoing dialogue 
with VIGIE2020/654 study colleagues, forming the IIF 3D Technical Specifica- 
tion Group, engaging even more widely with other specialists and representatives 
across user communities and standards bodies. 
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1 Overview 


IF (International Image Interoperability Framework) is a collection of open web stan- 
dards broadly used for presenting and annotating high resolution images and audiovisual 
content [1]. These standards are adopted by hundreds of GLAM (galleries, libraries, 
archives, museums) organisations worldwide [2], and permit the reuse and recombina- 
tion of digital assets across traditional digital institutional boundaries [3]. While these 
standards have been successful for 2D images and Audio/Video (AV) data, there is a 
growing interest in how they might be applied to 3D content. An example of this would 
be to combine shards of pottery from disparate collections into a single whole within a 
viewing experience. 

The IIIF 3D Community Group [4] has been cataloguing user stories and user needs 
[5], exploring various 3D workflows [6], and initiated a viewer comparison project [7] 
with major 3D developers and researchers, and ongoing dialogue with VIGIE2020/654 
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[8] study colleagues. The viewer project was collaboratively considering common chal- 
lenges and potential solutions to key areas, as part of forming the IIF 3D Technical 
Specification Group [9], for which the 3D group is engaging even more widely with 
other specialists and representatives across user communities and standards bodies. 

Some key questions have been around the conceptual framework needed to address 
the use cases for 3D scenes or digital dioramas, including by adding depth to the current 
2D IIIF canvas model, and by embedding one or more canvases within a 3D scene 
(e.g. multiple paintings or texts or music associated with a cathedral, temple or palace, 
held in separate collections, reunited with their original walls and interiors of a suitable 
building model). With growing user and institutional demands, technical developments, 
and examples of advanced research collection and integration of virtual resources (e.g. 
Sketchfab [10], MorphoSource [11], Exhibit [12], Mozilla Hubs [13]), there is pressing 
need for a technical specification to ensure interoperability and longer term sustainability. 
The VIGIE2020/654 study has more information about these needs. 

The plan for the IIIF 3D Technical Specification Group is to continue a collaborative 
approach to clarifying and specifying interoperable frameworks for 3D data, including 
common ways to: 


1. annotate 3D media of various types into a shared canvas space, with commentary 
2. combine 3D media with images and AV content within a shared space 
3. specify the presentation (placement, orientation, and contextualization) of 3D media 


The group will continue its work with other standards bodies and 3D image viewer 
developers, to collaboratively address the many challenges around this developing area. 
The combined and widespread expertise from the many 3D specialists will continue 
to guide the work of the INF 3D Technical Specification Group, as it outlines sus- 
tainable options for the interworking of existing open standards (e.g. VRML/X3D [14], 
WebXR [15]), established foundations (e.g. WebGL [16], Three.js [17], react-three-fiber 
[18]), and emerging proposals (e.g. <model> tag [19]), to provide recommendations 
for expansions to and modifications of IF APIs to better interoperate with the evolving 
digital ecosystem of online 3D content. 

These 3D developments will complement the ongoing updates and continued 
widespread adoption of the IIF technical specifications for 2D and AV data, which 
has been enabling greater access to widespread resources — of even greater significance 
for teaching and learning and research during the global pandemic — as well as pro- 
viding for richer presentations, close object inspection through deep zoom ranges, and 
shareable annotation of media using W3C’s Web Annotation Data Model [20]. 

IIF specifications also enable the recombination of long-separated parts of an orig- 
inal whole (e.g. missing sections or leaves in a digitised medieval manuscript, viewed 
with missing pieces contained in another digitised collection). INF adoption continues to 
enable effective sharing and support for the preservation of cultural heritage resources, 
whether as individual items or as combinations of media from one or more collections, 
locally and globally. 
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These technical developments for the community rely on regular meetings and input 
from the community. Planning for the IIF 3D technical specification includes recorded 
monthly meetings, complementing the more general 3D Community Group presenta- 
tions and discussions, and involving group problem-solving with regular input from and 
interactions with representation from 3D researchers and media and viewer developers, 
including from University of Cambridge, Duke University, University of Edinburgh, 
UC San Diego universities, Deutsches Museum, IIIF Consortium, Mnemoscene, Mor- 
phoSource, MPEG consortium, Sketchfab, Smithsonian Institution, Visual Computing 
Lab (CNR-ISTI, Italy), Web3D Consortium. Google has presented and been part of our 
deliberations, and we plan to follow up with Apple to learn more about the plan by the 
WebKit team to propose a <model> HTML element for 3D. We also intend to maintain 
a close engagement with existing web-based 3D standards such as X3D, with its VRML 
roots, and WebXR. 

We regularly collaborate with 3D-related projects and funding proposals, and ask 
funding bodies to help engage others in further developing this collaborative community, 
ensuring that we interconnect with more communities, to further develop 3D standards 
which will be the most widely usable and adopted across current and future proposals 
and projects. 


2 Key Use Cases and Prototypes 


Drawing on more than a year engaging with various individuals, institutions, and pro- 
fessions through regular IIIF and other group meetings and conferences, the IIF 3D 
Community Group identified a wide set of user stories [21]. With that input, and as part 
of the work of the group making preparations for the 3D TSG, the core practical use 
cases identified are: 


e Display a 3D model, specifying position, orientation, and scale 

e Display multiple 3D Models in a shared space 

e Display a 3D model alongside 2D images and AV content within a single viewing 
experience 

e Annotate displayed 3D models with commentary 

e Sharing camera position, orientation, and target 

e Display multiple 3D models that can be rotated and manipulated independently 


To clarify the viability of meeting these cases, some research and demonstrations 
were able to highlight the availability of existing practical options, as a basis to proceed 
and to indicate areas for further development in the future. For instance, there was interest 
in an initial demonstration of the sharing of two or more 3D models, stored in different 
formats, which could be rendered in the same visual space. This is a basic requirement, 
similar to the common rendering of differently formatted 2D images together (e.g. JPEGs 
and PNGs) in the same viewer, which is essential to enable the important IIIF options 
for interoperability across the necessary formats. 

While it has long been possible to manually load 3D models in a web browser, using 
something like the popular three.js cross-browser JavaScript library and the underlying 
WebGL application programming interface (API), there are well-developed viewers 
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built on such frameworks which provide additional options useful for interacting with 
the models. As part of the IIIF 3D group efforts, even more advanced options have been 
shown in some technical proofs of concept that demonstrate multiple assets in different 
formats being rendered in the same virtual space, most using IIIF in some capacity, in 
particular: 


e The Infinite Canvas [22]: Combines a 3D asset, an audiovisual clip, and multiple still 
2D images in a single navigable 3D space, all using IIIF manifests for each asset 
(Fig. 1). 


One Small Step (video) Perpetual Motion Machine 


by @edsilv 


Fig. 1. Figure from the initial screen of The Infinite Canvas, a proof of concept application which 
is shown displaying 2D, an audiovisual clip, and a 3D model. 


e Antikythera Mechanism [23]: Combines three distinctly different types of 3D files 
(GLTF, CT — computed tomography, and point clouds; uses JSON, not currently in 
IIF manifest format) (Fig. 2) 

e Mozilla Hubs gallery demonstrating three 3D assets from different sources [24]: 
created for the IIIF June 2021 Annual Conference to demonstrate that it is possible to 
combine 3D assets from different data resources (in this case MorphoSource, Royal 
Pavilion & Museums Trust [25], and The British Library [26]). All assets are hosted 
via IIIF and IIF was used to locate the assets. IIF manifests were not used directly 
at this point, however the actual models were downloaded from repositories and 
uploaded to Hub. In Mozilla Hubs, the space is navigable either on flat computer 
monitors, mobile AR, or in a VR headset, and multiple users can interact with each 
other and with the objects in the space (Fig. 3). 
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Fig. 2. Figure of the Antikythera Mechanism, combining three different types of 3D files - GLTF, 
CT (computed tomography), and point clouds. 


Fig. 3. Figure from a Mozilla Hubs gallery, displaying 3D assets from three sources — Mor- 
phoSource, Royal Pavilion & Museums Trust, The British Library; used with permission from 
Julie Winchester, Doug Boyer, Edward Silverton, and respective collection holders. 


3 Combining Content in 3D Environments 


To focus on the operational use cases for combining media in a 3D environment, and 
to consider best options to enable the intersection of 3D and non-3D content in the 
same rendering environment in a web browser, the IIIF group discussions and follow 
up research identified two main approaches for translating the traditional IIIF canvas 
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into a 3D scene. These are high-level and use case oriented solutions for creating HIF 
manifests that address some options for adding a third dimension to extend the existing 
2D canvas or otherwise fitting the 2D canvas in a 3D space. While not necessarily 
mutually exclusive, these options require further consideration and experimentation to 
help enable and encourage interoperable and sustainable development in the 3D context. 


Canvases and Scenes 
By default, the IIIF presentation specification makes use of a canvas concept which has 
some of the following basic properties: 


Bounded dimensions (width and height) 

Coordinate origin at top left 

Fundamentally “flat,” intended for standard mobile/computer 2D use cases 

Can place standard assets (images, video, audio) and 3D mesh models on the canvas 


— Standard assets have parameters to indicate size and position within the canvas 
e Can nest secondary (and subsequent) canvases inside a primary canvas 


— Placement of the secondary canvas is determined by parameters indicating size 
and position within the canvas 


We have proposed a kind of straw man concept, to compare and contrast with the 
canvas, calling it a “scene”. Scenes are similar to canvases but have some important 
differences. The following basic properties apply to scenes: 


Unbounded dimensions (across X, Y, and Z axes, or equivalent polar coordinates) 
Origin at the centre of the coordinate space 

Can place standard assets (images, video, audio), 3D mesh models, 3D point clouds, 
or 3D volumes in the scene 


— All assets have parameters to indicate scale (3D) or size (2D/AV), position, and 
rotation within the scene relative to the scene’s coordinate origin 


e Can nest secondary scenes within primary scenes 


— Placement of the secondary scene is determined by parameters indicating size, 
position, and rotation within the scene. Objects inside the secondary scene respect 
the coordinate origin of the secondary scene, but end up modified by the placement 
of the secondary scene inside the primary scene. Example: An overall “world” 
scene that has placed a single secondary “room” scene at position (0, 0, 10). The 
room has a lamp inside of it at the room’s origin (0, 0, 0). To observers, the lamp 
would be located at world coordinates (0, 0, 10). 
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e Can nest secondary canvases within primary scenes 


— Placement of the secondary canvas is determined by parameters indicating size, 
position, and rotation within the scene. 


Additionally, the scene concept requires one additional basic property to be added 
to canvases, namely that: 


e Scenes can be nested inside primary canvases 


— Placement of the secondary scene is determined by parameters indicating size and 
position within the canvas 


Therefore, canvases and scenes can nest inside of each other (and canvases can nest 
inside canvases, and scenes can nest inside scenes, etc.). It could be argued that canvases 
and scenes are really the same thing, but with different component-style properties (e.g., 
see the IIIF-ECS extension proposal [27]). These ideas could easily be expanded to 
treat canvases and scenes as specific manifestations of a more generic entity. This would 
make some of the operational use case examples even simpler to describe than they are 
explained here, at the expense of introducing more lower-level complexity. 

While there is a question of whether this proposal would entail a commitment to 
setting a 0,0 origin point in the top left corner of a canvas, with a z-axis extending from 
there outwards, there is also consideration that this would be unnecessarily complicated 
to implement compared to a common configuration of an infinite scene setting 0,0,0 in 
its centre. The work of the IIIF Technical Specification Group has been supported by the 
understanding that the Shared Canvas Model can potentially be altered to permit “scene- 
like” functionality. In order for 3D to become a useful part of the IIIF specification it 
will require the same level of changes and the resulting kind of core status as has been 
achieved with that addition of the IIF AV technical specification, as part of the major 
update of the Presentation API in 2021. 


Scenarios for Transforming 2D into 3D 
Extend 2D Canvas I: I want to display a single 3D object on a flat webpage for rotation, 
zooming, panning of the object. 

This is an operational use case that IIF 3.0 does currently support for 3D meshes, 
via the “Model” or “PhysicalObject” annotation types, although the standard parameters 
for size (height and width) don’t translate naturally to a 3D object and there is no way 
to specify properties such as rotation. 

Using the concepts described above, this use case would be achieved by a primary 
canvas nesting a single secondary scene, with the scene’s size dimensions equal to the 
size of the canvas. The 3D object would be placed within the scene at the origin with 
scale, rotation, and other properties explicitly specified. 

The currently supported IIF manifest that presents a 3D Model or PhysicalObject 
annotation as the only annotation on a canvas could easily be considered a simplified 
convenient way of describing the more complex setup described above. In other words, 
creating a IIF manifest that posits a canvas with a single PhysicalObject annotation 
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could be read as a convenience method to describe a Canvas -> Scene -> Physical 
Object (Scale 1, Rotation null, Position {0, 0, 0}) hierarchy. In fact, a potential strength 
of this solution is that it allows for “collapsing” or “expanding” complex circumstances 
as is needed. 

Extend 2D Canvas II: / want to display several 3D objects in separate views on a 
flat webpage so that viewers can compare and contrast multiple objects at once. 

This could be achieved with a single primary canvas nesting multiple secondary 
scenes, with the gridding and size of each scene determined at the canvas level. Each 
secondary scene would place a single 3D object at the origin with standard scale and no 
rotation. 

For reference, this approach is how the Aleph viewer [28] on MorphoSource uses IIF 
manifests currently, although it obviously goes beyond the current IIF specifications in 
order to display volumes as well as meshes. 

Fit 2D Into 3D Space I: J want users to interact freely (in flat, VR, or AR contexts) 
with a virtual space. This virtual space should have a mesh and a point cloud, bounded 
by two “walls” displaying 2D content that are already accessible via two standard IHF 
3.0 manifests. 

The primary entity in this operational use case is a scene rather than a canvas (or 
perhaps a canvas specifying flat screen size that nests the world scene within it). The mesh 
and point cloud are nested within the scene at the desired positions, scales, rotations, etc. 
Finally, the two already-existing IIIF manifests - each with its own respective canvas - are 
nested within the 3D scene with their own position, scale, and rotation. The placement of 
the assets within each canvas are determined by the original manifests for those canvases, 
and are just displayed in the world scene. 

Fit 2D Into 3D Space II: J want to fill a virtual gallery hall with individual virtual 
rooms. Each room has been designed by a separate group and they consist of a mix of 
3D objects displayed on room floors and 2D content displayed on room walls. 

Each room in this example is equivalent to the operational use case described above 
this, and so each room is a scene nesting combinations of assets and IIIF manifests 
describing canvases to serve as “walls.” Rooms are then nested as multiple secondary 
scenes inside a larger world scene, which provides the position/rotation/scale information 
necessary to connect the rooms together, make them sit alongside each other, etc. Objects 
in each room are placed using the coordinate systems of the rooms (or in some cases the 
top-left origin coordinate system of canvases inside rooms!), but the world scene brings 
these rooms together in a way that makes sense for the overall purpose of the project. 

For reference, this is the approach taken in Mozilla Hubs and the Infinite Canvas 
demonstrations noted above, with the latter also obviously going beyond the current IIF 
specifications in order to display the many media types. 


4 Requirements for Annotation in a 3D Environment 


The operational use cases for annotation in the context of IIIF and in a 3D environment 
have two fundamental purposes — to add textual and other forms of commentary to a 
3D model, and to add multiple models from different sources into a combined scene. 
The essential and broad use of annotation in a IIF context needs special considerations 
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for 3D environments, as highlighted in the following details from the IIF 3D Technical 
Specification Group. 


Annotations for Commentary 
The most basic form of 3D commentary annotation we can identify, specifies a point on 
the surface of a model by its coordinates (x, y, z). 

The point itself has no dimension, and serves as an anchor for a visual marker, often 
represented as a pin, or circular hotspot. This marker can either be displayed in screen 
space, overlaid on top of the 3D scene, or in world space, displayed within the 3D scene 
with its own geometry (such as a sphere). 

In the case of screen space annotations, they may always be visible regardless of 
position given that they are overlaid on top of the 3D scene. However, this is not always 
desirable, and a content publisher may want them to appear only when on a surface 
facing the camera. 

To achieve this, it is necessary for each annotation to store the surface normal vector 
of the mesh at the point where the annotation was created. This can be used to determine 
if the annotation is facing the camera and should be displayed, or if on the reverse side 
of a model and hidden. 

In the case of annotations located within the scene’s world space, it is not always 
possible to ensure that they are visible to the user just by defining their position, as the 3D 
model may occlude itself. For example, the arm of a statue might occlude an annotation 
on the torso, or if there are multiple objects within the scene one object might occlude 
another. 

Therefore it is necessary to optionally include a camera position and orientation 
along with the annotation coordinates in order for a content publisher to ensure that an 
annotation is visible to the user. If for example the user were using a simple previous/next 
interface to page through annotations, this would allow the camera view of the scene 
to be automatically adjusted for each annotation in order for it to be visible, without 
requiring the user to manually rotate the scene. 

The necessity to specify camera position and orientation per annotation is not limited 
to world space annotations however, and can also provide a satisfying user experience 
for screen space annotations. 

Beyond the concepts of screen space and world space, there is also “object space”, 
i.e. the coordinate system relative to each 3D model’s individual origin. This is a useful 
concept when loading models from various sources which carry their own commentary 
annotations, in that the annotations are positioned relative to each model. This bypasses 
the need to translate each model’s annotations into world space and means the models 
can easily be remixed in various ways without losing their associated annotations. 

While there are other forms of annotation possible beyond points on a surface, such 
as regions of a surface, or volumes within an object, we believe that focussing on the 
simplest case first will provide insights into how to extend into other modalities. 


Annotations for Scene Building 

Annotations with a “painting” motivation are used to combine images within a “shared 
canvas” coordinate space in IIIF. This concept could also extend to 3D models, where 
the body of an annotation is a 3D model URL, and the target is a canvas. 


Evolving Standards in Digital Cultural Heritage 59 


In order to create a 3D scene such as by placing artworks from various sources within 
a gallery space, or combining disparate shards of pottery into a whole, we will need the 
ability to specify xyz coordinates for each individual model within the shared canvas (as 
with commentary annotations). 

If for example each pottery shard was produced using a slightly different method, 
resulting in varying object scales, origins, and rotations, it will be necessary to over- 
ride/normalise these properties per annotation in order to combine them effectively. 
This is similar to how the Web Annotation Data Model allows images to be painted onto 
a canvas using a given xywh parameter. 

It is common practice in 3D viewers such as Sketchfab for users to specify an 
initial camera position and orientation for a 3D model or scene. This can satisfy an 
aesthetic or informative preference on behalf of the content publisher. This is similar 
to how commentary annotations require an associated camera position and orientation, 
and the same concept may apply to “painting” annotations. One could imagine a scene 
combining multiple 3D models, where clicking on each model animates the camera to 
a new position and orientation in order to best view that model. 


5 Authentication and Search APIs for 3D Content 


Given the ongoing developments with the Authentication and Content Search technical 
specification groups (TSGs), and the related changes introduced in version 3.0 of the 
IIF Presentation and Image APIs, as well as developments in the browser community 
and the evolving web landscape, the 3D Technical Specification Group expect similar 
challenges, as well as the need for collaboration with the other TSGs in these areas of 
mutual concern. 

The 3D TSG will coordinate efforts with those groups to ensure that any authentica- 
tion or search solutions pertaining to 3D content are either consistent with any updates 
from those groups, or can clarify and inform those groups about any changes which may 
be suggested by particular features of the broad base of 3D content forming part of the 
work of the 3D TSG. 

That said, and given the IIIF bases upon which the related 3D Community efforts 
are already developing (e.g. with reference to Sketchfab.com, and the open source tech- 
nologies used by MorphoSource.org), we anticipate no new difficulties with searching 
the text content of manifests and annotations to be associated with 3D data. Existing 
examples of annotations used with 3D data include key viewers used with Sketchfab, 
Smithsonian Institute, MorphoSource, and X3D content. Some annotation models are 
closer to existing IIF annotation approaches, and all annotation models will be key 
features of the 3D TSG’s further research and review of search-related concerns. 

Search and authentication requirements for 3D data may at some point (perhaps in a 
subsequent version rather than initial specification) include extra differentiation between 
component parts of a composite object, or with reference to individual items in a scene 
comprised of combined data which may require login (e.g. for selected images and 
audio-visual items). These will be matters for future endeavours, following the initial 
phase of 3D technical specification. 
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6 IIF 3D Technical Specification Group Charter 


The group effort to establish the HIF 3D Technical Specification Group expanded as 
it involved more committed institutions and volunteers, over the course of more than 
a year. That group includes research and cultural heritage institutions who are already 
publishing 3D content and who are willing to experiment with various approaches to 
help improve the creating, storing, cataloguing, sharing and sustaining of this media. 

The initial IIF 3D Technical Specification Group Charter, formally accepted by IF 
in December 2021, includes a list of those collaborating institutions and their represen- 
tatives, along with more information about planned scope and roadmaps and other key 
details. For reference, it is included here. 


HIF 3D Technical Specification Group Charter [29] 


Introduction 
As IIF has evolved from an initial focus on 2D images, encompassing new media 
modalities such as audiovisual content with time-based data, we see a plethora of unmet 
use cases throughout the cultural heritage field relating to the display of 3D media and 
related metadata. We recognise these requirements and the need for further developing 
a conceptual framework, which can complement and extend existing IIIF specifications. 
The IIF 3D Technical Specification Group will collaboratively clarify and specify 
common interoperable frameworks pertaining to the 3D data domain. This will include 
ways to: 


annotate 3D media of various types into a shared canvas space 

annotate 3D media with commentary 

combine 3D media with images and AV content within a shared space 

specify the presentation (placement, orientation, and contextualization) of 3D media 


The group will work with other standards bodies and 3D image viewer developers, 
and will collaboratively address challenges around this dynamic area, which shows great 
potential for a IIIF resolution, as practical options for media sharing and interchange, for 
which there is substantial demand and no demonstrably sustainable alternatives. Guided 
by widespread expertise from the 3D Community, committed to this purpose, the IHF 
3D TSG will outline sustainable options for the interworking of existing open standards, 
to provide recommendations for expansions to and modifications of IIIF APIs to better 
interoperate with the evolving digital ecosystem of online 3D content. 


Scope 

It is the intention of this group to explore suitable IIIF extensions and identify any 
necessary changes to the core IIF specifications (e.g. adding a third physical dimension) 
to support display of 3D media using IIF tools. After a period of time and suitable 
feedback from implementers, we expect to propose changes to the Presentation API, to 
accommodate stand alone and combinations of media, whether 2D, A/V or 3D. 
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This group will focus on the use cases identified, in versioned phases, including 


options to: 


display a 3D model, specifying position, orientation, and scale 
display a 3D model alongside a 2D image 

display multiple 3D models in a shared space 

annotate displayed 3D models with commentary 

specify initial camera position, orientation, and target in 3D space 


IMF 3D TSG will look forward to working with other IIIF groups, especially where 


there are shared areas of interest (e.g. museums, archives, maps), and welcomes contri- 
butions to collection of user stories, ongoing community discussions, and specialist app 
development. 


Deliverables 
The expected initial deliverables are IIIF API extensions and a specification change to 
define interoperable methods to enable: 


three-dimensionality via a third physical axis extending orthogonally from the tradi- 
tional canvas model, with accommodation for the scene concept, and consideration 
for use cases and backward compatibility 

viewing of 3D media, including combined media (multiple 3D assets and/or 3D and 
non-3D [2D or A/V] assets combined) 

asset positioning, orientation and scaling 

initial view, and shareable customised views 

adding associated annotations and linked metadata 


Roadmap 


Group formation December 2021 - (See: IHF 3D - TSG Preparation Checklist) 
Research and secure funding for development work — through December 2022 
Initial demo(s), testing, and feedback March 2023 

Broad set of prototype demos with testing and feedback June 2023 (Annual 
Conference) 

Draft specification change recommendations June 2025 (Annual Conference) 

Proof of concept specification implementation December 2025 (Fall Working 
Meeting) 


Communication Channels 


Github Repository: (https://github.com/IIIF/3d) 
Slack: #3d (https://iiif.slack.com/messages/3d; new accounts: http://bit.ly/iiif-slack) 
Email: iiif-discuss; subject line: [3D-TSG] 
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e Calls: complementing the 3D Community Group calls (see Community Calendar) 


Community Support 


e British Library (Adi Keinan-Schoonbaert) 

Cyprus University of Technology / Digital Heritage Research Lab (Marinos Ioan- 
nides) 

Deutsches Museum (Georg Hohmann) 

Duke University (Doug Boyer) 

ISTI - CNR (Federico Ponchio) 

MorphoSource (Julie Winchester) 

Mnemoscene (Ed Silverton) 

sketchfab.com (Thomas Flynn) 

The Smithsonian Institution (Jon Blundell, Jamie Cope, Vince Rossi) 
University of Basel (Peter Fornaro) 

University of Cambridge (Ronald Haynes) 

University of Edinburgh (Mike Boyd) 

University of Brighton (Karina Rodriguez Echavarria) 

UC San Diego (Scott McAvoy) 

University of Oxford (Kathryn Eccles, Nandy Millan) 


Technical Editors 


Rob Sanderson (Yale University) 
Mike Appleby (Yale University) 
Simeon Warner (Cornell University) 
Dawn Childress (UCLA) 

Tom Crane (Digirati) 


Feedback: iiif-discuss @ googlegroups.com 
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Abstract. Cultural heritage is an integral part of history and in order to better pre- 
serve, manage and highlight our cultural heritage it is necessary to first proceed to 
digitize and document it. Geometric documentation of a cultural heritage asset is 
the process of collecting, processing, rendering, and recording data to determine 
the location and actual form, shape, and size of a monument in three-dimensional 
space at a given time. This chapter presents the modern methods and technolo- 
gies for the 3D acquisition of cultural heritage assets and discusses the related 
challenges and risks. Moreover, we are presenting the possible quality aspects 
observed in the fieldwork that arise from the equipment, the environmental con- 
ditions that prevail during the surveying, the object itself, the personnel expertise 
and the stakeholder conditions. 


Keywords: Digital Cultural Heritage - 3D Acquisition - Quality and Complexity 
Aspects 


1 Introduction 


Digitization is now a fundamental step in the study and preservation of our cultural 
heritage (CH). The most common digitization product of tangible cultural assets is a 3D 
model, while the primary phase is the acquisition of the asset, whether it is a very small 
one or an entire archaeological site. The need to preserve a CH asset is directly linked 
to its documentation both for reasons related to its study and for reasons related to its 
promotion. One of the most important tasks at the stage of analyzing and documenting 
a monument is its digitization, i.e., the systematic recording and visualization of the 
elements that define the geometric form and the position in space of the individual parts 
of the monument at a given moment in time. 

In the strictly scientific sense of the term, digitization is a very specific process of 
the geometric representation of a real object, through observations made by measuring 
systems, using some suitable computational model which will return, in addition to the 
representation itself, and the possibility of assessing the accuracy and precision of the 
deliverable. This representation of the object being captured must end up in a medium 
of communication (eg a 3D model) to be usable by all the stakeholders. 


© The Author(s) 2023 
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https://doi.org/10.1007/978-3-03 1-35593-6_4 


66 S. Chatzistamatis et al. 


Digitization, as a process of representing a real object, depends on the accuracy 
and precision in determining the geometric elements of the object in the single three- 
dimensional space and the density (or spatial resolution) of the points that will be repre- 
sented to reconstruct its geometry. The combination of these three parameters (accuracy, 
precision and density) that determine the digitization, determines the quality of the rep- 
resentation. The type of surveying can be determined by the technique to be applied. The 
choice of the appropriate technology depends on a number of factors, which must be 
taken into account in choosing the procedure to follow. These factors concern the char- 
acteristics of the object to be surveyed, such as its shape, size, geometric complexity, 
surrounding space, cost and available time, and the possibility of applying new technol- 
ogy to the process [1]. The traditional techniques that are applied for the surveying of 
objects or large-scale sites are the topographic and the photogrammetric method. 

The development of technology in the imaging sciences in recent years has introduced 
new techniques for measuring space and objects. The introduction of laser technology at 
the beginning of the 60s and the understanding of the advantages of the characteristics of 
the beam it uses, such as monochromaticity, good alignment and the ease of shaping the 
laser beam, led from a very early stage to its application in measurement technologies 
and imaging. In recent years, the surveying of cultural heritage sites with 3D laser 
scanners has been added to classic methods. This chapter presents the modern methods 
and technologies for the 3D acquisition of cultural heritage assets and addresses the 
factors observed in the fieldwork that affect the quality of the final product. 


2 Technologies for 3D Acquisition 


Scanning instruments are distinguished, depending on their nature, into contact and non- 
contact. Contact scanners measure the object through physical contact, giving relative 
coordinates to an embedded recording system. These systems are not used usually in 
cultural heritage objects but mainly in industrial applications. The reason they are not 
used on cultural heritage objects is their main disadvantage, that is, they require physical 
contact with the object, which can lead to the change or even the destruction of the 
object. Heritage items are very sensitive and therefore this method is not suggested. 

On the contrary, non-contact instruments and methods are suitable for acquiring 
cultural heritage assets, as long as they do not emit a beam that can harm the asset. 
Figure | depicts the taxonomy of non-contact instruments used today to acquire cultural 
heritage objects. Choosing the correct method depends on factors, most importantly the 
size and complexity of the object. The authors in [1] clarify the term complexity of an 
object and argue that choosing the correct method is based on the complexity of the 
object and the complexity of the whole process. Such factors that influence the selection 
of the correct method are the availability of equipment, the expertise of the personnel 
and the budget. In the literature, and especially in large-scale digitization, we find the 
fusion or integration (see Fig. 2) of these methods [2-6]. 
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Taxonomy of 
technologies for 
3D acquisition 
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Aerial | 
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Global Navigation 
Satellite System 
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methods 


Airborne 


Terrestrial Terrestrial 


Vehicle-based 
mobile 


Handheld or 
Backpack mobile 


Fig. 1. Taxonomy of technologies for 3D acquisition of cultural heritage assets. 


2.1 Photogrammetry Methods 


Photogrammetry is the art, science, and technology of extracting 3D information from 
images. The process involves taking overlapping images of an object, structure, or area 
and converting them into two- or three-dimensional digital models. The basic principle of 
photogrammetry is if the directions of the same objects photographed from two extrem- 
ities of the measured base are known, their position can be located by the intersection 
of two rays to the same object. 

There are two main types of photogrammetry: aerial and terrestrial or close-range. 
Aerial is the process of using an aircraft to capture aerial images that can be con- 
verted into a three-dimensional model or digitally mapped. The most common type of 
aerial photogrammetry is photography by a suitable camera mounted on an Unmanned 
Aerial Vehicle (UAV). UAVs have facilitated the safe aerial photography of hard-to- 
reach objects or hard-to-reach areas where traditional surveying could be dangerous 
or impractical. Terrestrial or close-range photogrammetry is when images are captured 
using a hand-held or tripod-mounted camera. The result of this method is usually the 
construction of 3D models of smaller objects. 

The main advantage of photogrammetry methods is that they are cheap, but the 
disadvantage is that require a lot of processing of the images taken to create the 3D 
model. However, the evolution of computing power along with the evolution of computer 
vision algorithms have led to the possibility of an accurate 3D model in just a few hours 
even for large-scale sites [7]. Along with acquiring images, it is necessary to acquire 
control points that will help to “stitch” the images and create the model to the correct 
scale. 
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Fig. 2. Point cloud delivered by integration of technologies in “DigiArc” project (https://www. 
digiarc.eu); UAV photogrammetry (top), Terrestrial Laser Scanning and Handheld Scanning (coat 
of arms). 


2.2 Laser Scanners 


3D laser scanners are active imaging instruments that provide real-time coordinates of 
the object being captured in three dimensions. Their basic operating principle is that a 
laser beam is emitted and for each point in space its coordinates (XYZ) are recorded. At 
the same time, the scanner records the reflectivity of the point giving an intensity value 
(i). While scanning, modern scanners also collect images with their built-in camera to 
define the color and texture of the imaged surface. 


Fig. 3. Terrestrial Z + F! laser scanning in “DigiArc” project (https://www.digiarc.eu). 


A scan can be completed in a few minutes and give us a detailed point cloud that 
forms and renders the geometry of the object in great detail. All laser scanners work 
via line of site. This means that on a typical project multiple scans need to be taken 
from different vantage points to ensure a complete dataset. A change in the position and 
orientation of multiple scans must be applied so that each point cloud uses a common 
homogeneous coordinate system of reference. This process is characterized as cloud 
alignment or registration. This is done by using artificial targets (most common are 
checkerboards or spheres) in the scans or by allowing enough overlap in the scans to 
register by recognizing common features (cloud-to-cloud). 


1 https://www.zofre.de/en/laser-scanners/3d-laser-scanner/z-F-imagerr-5016. 
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3D laser scanners are classified in airborne (known as LiDAR-Light Detection and 
Ranging), terrestrial (see Fig. 3) and mobile. Airborne scanners are equipped with sensors 
to measure the position, orientation and altitude of the aircraft during data collection, 
through inertial sensors [8]. Combining these measurements with distance data collected 
by the laser scanner produces a 3D point cloud representing the topography of the 
ground, similar to that generated by a terrestrial laser scanner. Mobile laser scanners are 
distinguished into those that are vehicle-based and those that are handheld or backpacked 
[9]. The latters find application in the acquisition of large archaeological sites where, 
due to the peculiarity of the terrain, the use of a vehicle is impossible. 

Based on their operating principle, laser scanner systems are divided into distance 
measurement scanners and triangulation scanners. Scanners of the first category can 
measure distance in two ways: a) Time of Flight (Tof) where a laser beam is emitted at the 
object and the distance between the source and the reflected surface is calculated from the 
travel time of the signal between emission and reception, and b) Phase Comparison where 
the emitted beam is converted into a harmonic wave and the distance is calculated from 
the phase difference between the transmitted and received wave. Triangulation scanners 
operate under the principle where a transmitter sends the laser beam at a specified variable 
angle from the edge of the mechanical base to the object and a camera at the other end of 
the base locates the laser spot on the object. The three-dimensional position of the object 
is carried out by solving the triangle formed. Triangulation scanners are only applicable 
to close distances and small objects as the accuracy of the distance between instrument 
and object depend on the square of the distance. 


2.3 Handheld for Close-Range Acquisition 


These are portable, small-sized devices that scan small objects with high precision and 
use the principle of triangulation or the principle of structured light. At the same time, 
passive receivers capture color data of the object in order to create a more complete 
model. Scanning with handheld scanners is usually aided by marking on the object 
certain points with the help of self-adhesive reflective tapes (as with the use of artificial 
targets). 

Structured light is a technology where an infrared emitter emits structured light, i.e. 
a pre-designed pixel pattern. This is non-visible light (near-infrared wavelength), which 
passes through a filter and is scattered into a semi-random but constant pattern of small 
dots projected into the environment in front of the sensor. The depth sensor receives the 
reflected pattern and calculates the shape and position of the object. The combination 
of depth information and color information from the camera is combined to create the 
point cloud. 


2.4 Supplementary Instruments 


Using control points in a survey can serve two purposes. The first is to provide a network 
of precise points so that point clouds can be successfully identified in a common coor- 
dinate system. Registration using cloud-to-cloud techniques can be very successful, but 
around a larger structure, for example, or where it is difficult to maintain a good overlap, 
control points are essential. It also provides certainty and, with some redundancy in 


70 S. Chatzistamatis et al. 


the network of points, an estimate of overall accuracy. The second possible purpose of 
control points is to relate the network of points for the object being surveyed to either a 
wider coordinate system or a known coordinate system and altitude data. 

A very accurate coordinate system can provide the basis for long-term tracking or 
structural analysis. For large-scale sites, a network of points associated with a known 
coordinate system can also provide tracking service over a wider area. The georeferenc- 
ing of the surveyed monuments can lead to studies of their spatial relationship and a more 
extensive archaeological analysis. An important aspect of the data collection process is 
defining a network of control points to which all other field metric data can be referenced. 
The specifications set are to require full georeferencing in a known coordinate network 
throughout the site. Also, control points are very likely to be needed locally to identify 
scans, for example, of the interior and exterior of a structure. 


Fig. 4. Measurements with Total Station (left) and GNSS (right) in “DigiArc” project (https:// 
www.digiarc.eu). 


To create a network of high-precision control points, measurements are required 
mainly with two instruments: a Global Navigation Satellite System (GNSS) that works 
with the principle of Real Time Kinematic (RTK) method and a geodetic station (Total 
station) (see Fig. 4). RTK is a kinematic determination, in which two receivers (base - 
rover) are used, provided that there is communication between the receivers, which is 
carried out either with a UHF modem or a GSM modem. The mobile receiver continu- 
ously receives corrections from the base, which has known X,Y,Z coordinates, and uses 
them to resolve errors from the satellite imagery to achieve greater horizontal and altitude 
accuracy at the measurement locations. With this method, an accuracy of the centimeter 
can be achieved almost in real-time. A Total Station is a topographic instrument capable 
of measuring angles and distances. It combines in a single device, a digital theodolite 
with electronic distance measurement (EDM) to measure both vertical and horizontal 
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angles and the slope distance from the instrument at a specific point, and an onboard 
computer for data collection and performing triangulation calculations. It allows its user 
to collect all the measurements, especially control points in remote areas, necessary for 
a topographic survey using digital technology. 


3 Factors Affecting the Quality of 3D Acquisition 


The quality of surveying and their final product depends on a range of parameters that 
do not only concern the accuracy of the instrument and the measurements. Factors such 
as the environment, the characteristics of the object to be scanned, and the methodology 
followed in each surveying process can affect the quality of the final result of the survey 
[10]. These factors can be distinguished (see Fig. 5) into those related to the operation 
of the instrument, the form and nature of the object, the environmental effects and the 
choice of methodology during the measurement process [11, 12]. 


Taxonomy of 
factors affecting 
the quality 


object materials environmental 


see human factor 
and geometry conditions 7 


instrumental 


Fig. 5. Taxonomy of factors affecting the quality of the 3D acquisition of cultural heritage assets. 


3.1 Instrumental 


These factors depend on the type of instrument and mainly affect the accuracy of the 
calculation of the measurements of distances and angles of the laser beam. The dimen- 
sions of the laser beam spot greatly affect the resolution of the generated point cloud and 
the determination of the position of the points. The greater its width, the more chances 
there are to create discrepancies in locating the coordinates of the points to be captured. 
The shape of the laser beam affects in the same way, which tends to grow at a distance 
from the instrument [13] and presents a dispersion effect. This factor is known as “beam 
divergence” and it is a specification provided by the instrument’s manufacturer. When 
surveying, the specification of beam divergence must be considered when positioning 
the instrument (see Fig. 6). New instruments tend to allow multiple measurements of a 
point leading to better measurement. 

Another factor that can affect the quality of the point cloud and is directly related to 
the laser beam is the edge effect. This factor is observed on the edges of the captured 
objects. When the laser beam reaches a point that is at the edge of the desired object, 
only a part of it is reflected and returned to the scanner. The remainder may be reflected 
from a surface behind the edge, if present, from an adjacent point, or not at all. Thus, 
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Fig. 6. Beam Divergence. 


signals from different areas return to the scanner. Defining the position of the point is 
done by averaging these measurements, causing the edge of the object to appear in the 
wrong position. The error, in this case, can range from fractions of a millimeter to a few 
centimeters. The latest updates of software packages that accompany the instruments 
can correct some of the edge effect mismeasurements. 

Most scanners use mirrors to deflect the laser beam in a specific direction. Any 
deviations created in the angle created by the emitted beam and the mirror surface can 
lead to the calculation of incorrect point coordinates. Errors related to the geometric 
stability of the scanner axes also lead to the definition of the wrong point position. 
The geometry of the scanner is described by three axes, the vertical, the horizontal and 
the alignment axis. The vertical axis is the axis of rotation of the scanning head, the 
horizontal, the axis of rotation of the mirrors and the third is the axis defined by the laser 
beam. These axes if are not perfectly aligned, errors may occur in the measurements 
of the angles formed by the alignment axis and the horizontal (alignment error) and by 
the horizontal and vertical (horizontal axis error). The instrument has a specific time 
of operation before calibration is needed. Referring to the manufacturer’s specifications 
can indicate the time that the instrument needs calibration. 


3.2 Object Materials and Geometry 


The different characteristics of the surface of the objects to be acquired can affect the 
accuracy of the point distance measurements, e.g. from the scanner, which depends on the 
reflection of the laser beam. Depending on the material, color, roughness, temperature 
and humidity of the surfaces of the object being acquired, the reflected beam shows 
variations. For example, on surfaces characterized by high reflectivities, such as white, 
distance measurements are more reliable than those with less reflectivity, such as black. 
This happens because white surfaces absorb less of the visible radiation and thus the 
intensity of the reflected ray is high, in contrast to black surfaces, which absorb more of 
the radiation, and the intensity is weak. 
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If the reflectivity of the object is too high or too low, measurements may not be 
made or show unacceptable accuracy. Corresponding error results also arise in cases of 
translucent material surfaces, in which the refraction of the laser beam and its reflection 
on the surface itself is observed (see Fig. 7). Refraction is the change in direction of 
propagation of a wave when the wave passes from one medium into another and changes 
its speed. Reflection of light occurs when the waves encounter a surface or other boundary 
that does not absorb the energy of the radiation and bounces the waves away from the 
surface. 


' normal 


incident beam 


reflected beam 
angle of 


incidence _} 


i angle of 
| refraction 


refracted 
beam 


Fig. 7. Reflection and Refraction effect. 


Other object characteristics that affect the results of point distance measurements are 
size, curvature, and orientation. A typical example is when we have surfaces where the 
effects of refraction and reflection are observed, leading to incorrect measurements [14]. 
Optical properties of materials such as glass windows or water (e.g. in a fountain) are a 
common aspect that affects the quality of the distance measurement, due to the refraction 
effect. The surface reflection on a mirror of a laser beam normally causes reflected beams 
in many directions. A solution is to cover these surfaces during the surveying process. 


3.3 Environmental Conditions 


Environmental factors such as atmospheric temperature, pressure, rain, humidity, and 
vibrations affect the accuracy of measurements. The presence of wind can cause the 
presence of dust, which can lead to incorrect measurements whether it is a laser scanner 
or photogrammetric techniques. For example, temperature changes during the process of 
a survey affect the speed of the laser beam, which is highly dependent on the density of 
the air. This results in reducing the quality of its measurement. Also, possible radiation 
interference from external light sources, such as a projector or sunlight, can alter the 
measurement results. In the case of aerial photogrammetry, the correct time of day must 
be chosen so that there is adequate lighting, as shadows greatly affect the photography, 
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and thus the final result. Another factor that affects the measurement results is the move- 
ments of the instrument in relation to the object being captured. Throughout the scan, 
the scanner must be stable to avoid erroneous data collection. Photographing an object 
with a drone can be affected by the presence of wind. All technologies are accompanied 
by technical specifications concerning environmental conditions, which must be taken 
into account and strictly observed during the surveying process. 


3.4 Human Factor 


Even if the three previous categories are considered, the human factor always comes 
into the "equation" of quality. The personnel expertise and therefore the methodology 
they will follow can affect the quality of the deliverables. These factors are related to 
the methodology chosen either in the phase of defining the specifications during the 
planning process or the surveying process and in the processing of the generated data. 
Such factors can arise from a poor choice of settings such as the desired resolution of the 
sampling and the distance from the object to be acquired or can result from the wrong 
approach to georeferencing the point cloud. Thus, during the planning of a survey, critical 
questions must be answered, such as choosing the correct technology or a combination 
of technologies. For the case of scanning with a scanner, the correct resolution, the use 
or not and the correct placement of artificial targets, the overlap or not with a percentage 
of at least 30% must be taken into account for the correct registration of the point clouds 
with the cloud-to-cloud method and the placement of control points to control the quality 
of the produced product. In the case of photogrammetry, the correct choice of camera 
and lens, the desired resolution of the images, the flight plan, the correct orientation of 
the camera (oblique, nadir), the overlap of the photos at least 80% and the use and the 
placement of control points to assist further processing and control the quality of the 
product produced [15]. 


4 Conclusions 


This chapter presents the latest technologies and systems available for the 3D acqui- 
sition of tangible cultural heritage assets. Each of these technologies has advantages 
and disadvantages, while many times in the surveys, especially of large-scale sites, a 
combination of these technologies is required. Choosing the correct method for a survey 
depends on factors such as equipment availability, personnel expertise, budget, and item 
complexity. However, despite the development of technology, many factors can affect 
the quality of the acquisition and thus of the final product, which must be taken into 
account during the planning and the process of surveying, so that the metric information 
is accurate. Along with the evolution of the instruments, the accompanying software for 
processing the received data is also evolving. These software packages, if updated, can 
algorithmically reduce many of these factors that affect the quality. 
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Abstract. 3D models from passive muted subjects, often used in the books and in 
preservation design reports as powerful images dense of contents, have nowadays 
the opportunity to become ’live gears’ leveraging knowledge, interpretation, and 
management into preservation objectives till to better-informed fruition. To this 
aim, we need to build up reliable and re-usable 3D Quality models. How to shift 
from a 3D model toward a 3D quality model? 

This contribution intends to focus on the parameters defining a 3D Quality 
model catching the heritage complexity with its components in a holistic method- 
ological and practical vision. A radar chart has been used to manage all the param- 
eters. First of all, Geometry describes a quality model: parameters for data acqui- 
sition, on-site surveying, and model processing to obtain 2D-3D Geometry quality 
are defined. The concept of scale associated with measurable parameters defining 
the Grade of Accuracy is proposed and applied to the surveying and to the 3D 
models. 3D models can be considered tools to decode the complexity of cultural 
heritage made by the different transformations across the centuries, anthropic- 
natural hazards, climate change threats and events (such as earthquakes, fires, 
wars). Thus, Geometry is not enough to describe such complexity; it represents 
the first step. Materials and Construction technologies analysis is the second pil- 
lar qualifying a quality model. The connection with the indirect data source (i.e., 
historical reports and archives documents), is the third pillar to be reconnected 
to the Geometry and Material analysis in the quality definition. HBIM represents 
a multidisciplinary environment to convey the information related to geometry 
and models. Furtherly, several parameters are identified to describe the quality of 
informative models, as in the case of Object Libraries and Building archeology pro- 
gressively feeding such models. BIM Level of Developments (phases) and Level 
of Geometry (contents, not scale!) have been adapted to the HBIM, introducing 
digitization, surveying, and HBIM modeling into the preservation process. Finally, 
a quality model is defined by the capability to be re-used circulating Information 
and Models among the end-users as in the case of informed VR/AR through CDE 
and XR platforms. 


Keywords: 3D Quality models - Complexity - HBIM - Heritage Building 
Information Modelling - construction systems - monitoring - Scan-to-HBIM - 
modeling - generative modeling - Scale - GOA - LOD - LOG - building 
archaeology - 3D volume stratigraphy - CDE - VR/AR/MR - XR 


© The Author(s) 2023 
M. Ioannides and P. Patias (Eds.): 3D Research Challenges in Cultural Heritage IN, LNCS 13125, pp. 77-102, 2023. 
https://doi.org/10.1007/978-3-03 1-35593-6_5 


78 R. Brumana 
1 Introduction 


The digitization processes need a systemic approach, generating holistic multi-faceted 
models: ‘live’ digital twins continuously growing within a complex built environment, 
under the hazards, natural anthropic pressures as climate change, earthquakes and at the 
same time inheriting the complexity of the past. 

It can be helpful to remember the deep meaning of the term “complex” from 
Latin “complector”, hold tightly, and, metaphorically, gird, embrace, and comprehend. 

Edgar Morin “From the Concept of System to the Paradigm of Complexity” [1]: 
“In order to make sense of the concept of system, we must postulate a new, nonholistic 
principle of knowledge. However, this will be possible only if we conceive of systems in 
general and generic or generative terms- that is, in terms of a paradigm. A paradigm 
being defined here as the set of fundamental relations of association and/or opposition 
among a restricted number of master notions-relations that command or control all 
thoughts, discourses, and theories)”. 

“... Finally, there is the impossibility of understanding a complex mental structure 
from the wall of a reductive or simplifying mental structure (while the inverse is possi- 
ble)”. Morin asserts that to know is to separate, to analyze and reconnect to synthesize 
or make complex, to ‘complexize’. 

What we can derive, among others, from this assertion is that complexity need to 
be intended as a circular process, where we can separate and filter some aspects to 
understand some aspects: but they require to be reconnected at a further step; as well as 
we can’t simplify at the beginning but at the end. 


2 Complexity and Quality: #Parameters into a Multiple Variable 
Radar Chart 


3D quality models can help in driving complexity: moving from their ‘limbo’, between 
media, society of image and big data circulation thanks to the technological potential 
offered by digitization; moving from a substantial under-use toward their re-use sup- 
porting preservation analysis and activities during the extended life cycle maintenance, 
and data transfer toward a better communication rising awareness through informed 
contents. 

To become subjects of knowledge, they need a qualitative leap addressed to content 
creation-content use-content transferring-content re-use. 

Objectives of 3D quality models can be summarized as follows: 


e to better understand the current geometric asset (addressing surveying methods and 
3D models to embodying and registering the richness of geometries, anomalies, out 
of plumbs, profiles comparison at the different levels, different alignments of the 
portions of the construction periods); 

e to support the assessment and the data interpretation, understanding and connecting 
all the different data (i.e., materials, decays, different transformation phases, global 
and punctual behavior, including the structural behavior); 

e to drive the preservation plans starting from understanding and interpreting state of 
the art to drive the design and decision making; 
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e to prevent future damages fostering the planned preservation (Long Life Cycle 
Management and Maintenance); 

e to deploy knowledge transfer of enriched models among different operators; 

e topromote the re-use and circulation of knowledge sharing for dissemination purposes 
(MR/VR/AR). 


Different parameters (#parameters) describing the quality of surveying till to the data 
management and re-use of the informative system (HBIM) are hereafter identified. This 
paper is part of the result of VIGIE expert member’s research on the “Study on quality 
in 3D digitization of tangible cultural heritage” [2]: a radar chart has been set up by the 
coordinator to manage all the parameters identified by the different contributions to the 
multiple tasks [3]. 

The proposed radar quality diagram is here applied in the following paragraphs to one 
of the selected success story case studies (the Basilica di Santa Maria di Collemaggio), 
considering the multi-variables crossing surveying, processing, and data transfer, when 
facing the heritage complexity. Such parameters are hereafter described to measure the 
contents enriching the quality model: geometry, model scales, grade of accuracy, materi- 
als, construction techniques, informative systems, content transfer, building archeology, 
level of development and level of geometry, Long Life Cycle management through 
Common Data Environment. 


2.1 A Success Story into the Radar Chart: The Basilica di Santa Maria di 
Collemaggio (L’ Aquila, Earthquake 2009) Reopened to the Public in 2018, 
Awarded by Europa Nostra 2020 


The project ‘Re-start from Collemaggio’ is here considered to derive the lessons learned 
crossing the digitization process finalized to the preservation after the earthquake, helping 
to define the concept of quality and complexity when dealing with a challenging con- 
text and a stratified architectural document. According to the stakeholder requirement 
(the Superintendence Office), the generation of an advanced HBIM based on reliable 
surveys to address decision-making by the different actors involved in the preservation 
process has been undertaken [4]. This case study has been used to describe the param- 
eters proposed to measure the quality and results obtained. The Italian multinational 
energy company (EniServizi), after the earthquake occurred at L’ Aquila in April 2009 
(Fig. 1), launched the project "Re-start from Collemaggio” funding around 16 million 
Euro to preserve the damaged Basilica. HBIM model of the Basilica of Santa Maria 
di Collemaggio supported the preservation design project and the restoration after the 
earthquake (L’ Aquila, 2009). 

The Basilica of Santa Maria di Collemaggio received the European Heritage 
Award/Europa Nostra Award (2020, May 07, | Conservation | Italy | L’ Aquila), [4]. 
The Basilica attracts about 30,000 people for the Forgiveness Feast Day (Festa della 
Perdonanza) on 28-29th August, established by Pope Celestino V (before the Jubilee - 
‘Giubileum’ - set up by Pope Bonifacio VIII recurring in the world every 25 years). The 
Feast is celebrated with the procession ceremony to the Holy Door of the Basilica: it 
represents an extraordinary unicum of tangible and intangible values. The design and the 
preservation have been intended to transfer them to the future. The Basilica successfully 
reopened to the public in December 2017. 
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Fig. 1. The area struck by the earthquake at L’ Aquila and the damaged Basilica di Collemaggio 


3 How to Define the User Needs? #Scale as a Parameter 


User needs requirements are fragmented depending on many factors and circumstances, 
including funding availability. The decision to choose a high-medium-low level of pre- 
cision and accuracy depends on many factors. It requires to be coherent with the state 
of art. It needs parameters to address the surveying since the starting phases. But some- 
times, even if we are in a complex contest needing a high level of precision, due to the 
state of the art, the decisions are not or can’t be coherent and consistent. 

Thus the decision among high-medium-low level of details and contents requires to 
be transferred to the users with transparency to avoid mismatches during the use and re- 
use. Models, once generated, need to be circulated and re-used, sharing all the knowledge 
and Information. Information are progressively growing within the multidisciplinary 
preservation process, supporting sustainable interventions. 

Models are expected to support users, experts and non-experts to extract content. 
Models need to become a live instrument to accompany all efforts to transmit cultural 
heritage to future generations, throughout the preservation of all its materials together 
with construction technologies, considered in their physicality, a vehicle of immaterial, 
tangible and intangible contents. 

Given that object models are generated by different professionals for different uses 
(with different required precisions and details), this paper intends to identify the param- 
eters describing a quality model rising situational awareness among the experts and 
operators in the re-use of such models. 

Thus, the adoption of simplified models needs to be declared in order to avoid 
faulty inferences when comparing or re-using libraries of objects modeled with different 
precision and accuracy. 


3.1 How to Measure Quality Model? Surveying to Digitization: Scale 
as a Parameter to Check Quality Control and to Re-use Models 


#Scale parameter. A feasible model is based on a reliable data acquisition starting from 
the on-site surveys! Model scale can be a parameter to measure the quality: to this aim 
the HBIM Grade of Accuracy (GOA) is here inherited and associated to the different 
scales; the concept of scale can be considered as a switch to guarantee the user needs 
freedom of choosing the 3D quality model most appropriate to their different objectives; 
at the same time, to ensure a coherent use of such models being the users aware of the 
low-medium-high level of accuracy (scale) of the data collected and processed. 

A 3D quality content model starts from the proper identification of the instruments 
and methods to acquire and process the data. The scale concept can be considered 
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a parameter for selecting the proper instruments (precision) and reliable processing 
to obtain reliable results (accuracy). The concept of scale is commonly adopted by 
professionals (i.e., architects, engineers), even if qualitatively. The higher is the scale, 
the higher is the level of detail. But the scale concept has a robust meaning, mostly 
unknown: it allows to handle measurable parameters under a qualitative and quantitative 
point of view. Scale requirements have been adopted for many decades in the tenders’ 
specifications at a worldwide level. As it is in the case of the generation of technical maps 
at the cartographic scale: the scale 1:25.000 has been adopted for territorial maps (i.e., 
IGM maps by the Istituto Geografico Militare/Geographic Military Institute), the scale 
1:10.000 and 1:5000 has been adopted for the technical regional-scale maps, 1:2000- 
1:1000-1:500 for the municipality technical maps. The specifications are standardized 
in the aero-photogrammetric process in the cartographic domain and inherited by the 
architectural surveying specifications (i.e., 1:100—1:50—1:20-1:10-1:1). 


3.2 3D Quality Content Models: #scale as a Parameter to Select the Proper 
Instruments and to Process Reliable Data (Precision and Accuracy): 
Specifications, Scales of Representation, Minimum Detail, and Tolerance 
Parameters 


#Scales parameters: minimum detail and tolerance. Several specifications have been 
fixed to guarantee the required accuracy in the map generation since the starting phase 
of the data acquisition through the data processing. The two parameters conventionally 
adopted to measure the scale are (i) the minimum level of detail (the so-called Graphic 
Error, G.E. fixed = 0,2mm for all the scales) and (ii) the related Tolerance (T) = 2 + 
3 G.E (Table 1). The choice of the ‘scale’ depends on the survey’s objective and on the 
use of the final product: such parameters allow to identify the proper scale, the proper 
instruments, and methods of processing, to validate the output and to re-use the output 
(Fig. 2). 

For example, at 1:50 scale, the G.E. value is 1 cm, and 2,5 cm the tolerance (T). In the 
case of photogrammetric restitution (as rectified images or orthophoto), the minimum 
detail of the data acquisition is fixed at half the G.E. value at the given scale, with a 
restrictive requirement in order to consider the processing resampling. 


# 3D Model Parameter (Grade of Accuracy). The concept of Grade of Accuracy 
(GOA) has been related to the scales (GOA10, GOA20, GOA50, GOA100) and proposed 
to describe also the 3D object models quality [4], thus defining the minimum detail and 
tolerance values in the modeling generation (Fig. 2). 


The Grade of Accuracy of the different scales is associated with G.E and T parameters 
[5]. The identification of the surveying instruments and restitution scale needs to be 
coherent with the correspondent parameters to make them reliable for the different uses 
and user needs (Table 2): 3D Geometry Scale and Image Scale are defined through 
the given parameters (Fig. 3). The precision of the surveying, as in the case of cloud 
scans acquired with TLS (i.e., laser scanner FARO Focus 3D), allowed the extraction 
of vertical and horizontal profiles with high accuracy (2 + 5 mm), thus coherent to the 
scales GOA20- GOA50. Even if we assume a 1:50 model scale, the precision related to 
the surveying method, as profiles for geometric analysis or out of plumbs analysis, is 
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Fig. 2. Scale as a parameter of surveying, 2D drawings, 3D models, and HBIM: it is inherited 
from the technical cartographic maps specifications, here applied to the 3D model objects 


relative to 1:20/1:10, and it will joke in favor of the object model that will be generated. 
Viceversa, Mobile Mapping Systems (as ZEB GEOSLAM), very useful for massive data 
acquisition, have a precision coherent with a scale range 1:100—1:200. 

Using the conventional definition of Graphic Error fixes the smallest detail that can 
be represented at a given scale (G.E. = 0,2 mm), and of the related Tolerance (T = 2 
+ 3 GE. value), we can apply such values to all the different model scales obtaining 
indicators of precision and of interchange in the usability and data sharing among actors 
and experts. As an example, the mosaic floor of the Basilica of San Marcus has been 


realized at a 1:1 scale to support the maintenance intervention on the single ‘tessera’ 
maintaining the 3D waved shape [6]. 
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Table 1. Scale as a parameter to measure surveying, 2D drawings, 3D models, and HBIM: it is 
inherited from the technical cartographic maps specifications and here applied to the 3D model 
objects associating the Grade of Accuracy to the scale concept. 


Survey Minimum Detail | Minimum detail Tolerance Value | HBIM models 
Drawing Graphic Error in case T=2+3GE generated at 
3D model | (G. E.) of image/ raster data different scales 
and = 0,2 mm (G.E.) GOA 
HBIM = 0,2 mm/2 Grade of 
SCALES Accuracy 
1:1 0,2 mm 0,1 mm 0,4 + 0,6 mm GOA 1 
1:10 2mm 1 mm 4 — 6 mm GOA 10 
1:20 4 mm 2 mm 8 — 12 mm GOA 20 
1:50 10 mm 5mm 20 + 30 mm GOA 50 
1:100 20 mm 10 mm 40 + 60 mm GOA 100 
1:1000 200 mm 100 mm 400 + 600 mm GOA 1000 

3D quality content models DRAWINGAND | ERROR 


VERY 
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Fig. 3. Geometry and image scale are parameters of 3D quality models: their components can be 
acquired and processed at different scales. Courtesy of ABClab-GIcarus (F. Banfi, R. Brumana) 


The proposal is to apply the surveying specification concept, traditionally linked to 
the representation scales, also to the 3D quality models, and to the HBIM model objects. 
GOAs are proposed introducing Levels of Accuracy linked to the scale measurability, 
inherited from the BIM logic and adapted to the heritage preservation purposes, referring 
to the richness of model details. In the modeling generations, GOA20 model accuracy 


has been adopted for vaults components in the Basilica di Collemaggio, GOA10 model 
for the octagonal columns, etc. 


84 R. Brumana 


Table 2. The results obtained by the surveying: instruments precision, methods, and accuracy as 
parameters for reliable results of the 2D-3D output (restitutions of plans, sections and fronts) 


Surveying methods 


Required scales and Survey 


Accuracy (mean square error, 


and output methods s.q.m.) 
Few quality figures... 

Geodetic Network 27 stations o= +1mm 
Total Station Leica T70 

Geodetic Control Points 260 points o= +15mm 


Total Station Leica T70 (GCP 
for Scan REGISTRY and GCP 
for SFM images) 


Laser Scanner Faro Focus 3D 


182 point clouds (with 
geodetic network and GCP 
data registry) 


o= +3mm 


Direct hand survey of the 
column stones ashlars 


n. 14 Columns geometry 1:10; 
tot. 574 stone ashlars (9 = 13 
column courses at 1:2 scale) 
TOT ~ 53 m’ 


Photogrammetric image 
blocks 

Ortophoto / 3D models 

(i.e., external and internal 
walls surface vaults intrados) 


Ground Dimension Pixel 
(GDP) - Terrain Pixel 
Resolution: 

5mm (external fronts 1:50); 
2mm (internal fronts, north 
damaged wall and main façade 
1:20); 

1mm (vaults intrados 1:10); 
0,5mm (the main facade 1:5); 
TOT ~ 7.000 m? 


5mm (1:50) 
2mm (1:20) 
Imm (1:10) 


Plans (horizontal sections) 


1:50 Ground level (3000 m2), 
underground, crypt, first floor 
1:20 walls profiles for 3D 
model analysis 

1:5 columns (out of plumbs) 
TOT ~ 4200 m? 


o= +3mm 


Vertical Sections (transversal 
and longitudinal sections) 


1:20 - 12 sections with double 
view direction (and fronts 
integration) 


o= +3mm 


UAV Falcons 
Covering and facades 


RGB (GDP 10 mm-5mm) 
1:100-1:50 
IRT flights (GDP 1 cm) 


TOT ~ 3000 m2 
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4 Beyond Geometry Through Geometry: # Parameters to Manage 
Informative Enriched 3D Quality Models Contents Transferring 


# 3D Geometries. Geometries can be intended as the result of the wealth of details 
ensured conjugating massive data acquisition with high accuracy; advanced processing 
methodologies supporting the representation of the complex geometries are capable of 
shaping the uniqueness of the architectural complex and its components. 

As explained in the previous paragraph, it requires an advancement on the specifi- 
cations to identify the proper model accuracy, scale, and richness of details to decode, 
analyze and represent heritage complexity as a whole and, at the same time, in its compo- 
nents (i.e., structural elements, vaulted systems, columns, walls, till to the decorations, 
frescoes and stuccos). 3D models, once generated, start circulating to be re-used and 
shared by common data environments and platforms: thus, they need to be validated not 
just on the surveying side but also on the modeling and processing side. AVS available 
tools can be easily used. 

But precision is not enough! 

Models can be enriched by different parameters going beyond the geometry but 
intrinsically connected to the geometry. Hereafter, a list of parameters contributes to 3D 
quality models till to the informative system enrichment. 


# Generative Modeling, HBIM Enabling Models. The generative model process is 
a crucial parameter to get the Object models at the required scale starting from reliable 
surveys. Proper procedures of NURBS generative modeling contribute to guarantying the 
interoperability among pure modelers and parametric modelers, avoiding simplifications 
derived from not fully enabling models, enhancing capacity building, and upskilling 
among users and modelers (Fig. 4), [7]. 

The quality of a HBIM model depends on many parameters: the capacity of the model 
to embed geometry, images, radiometric information. It requires proper processing to get 
NURBS based Material (and Decay) Mapping as illustrated in the following paragraph, 
in order to support data management, and related metric computation (i.e., Area, Volume) 
together with all the information. 


4.1 Not Just a Matter of Measuring: Geometries Embodying Materials 
and Construction Techniques Through HBIM 


Not just a matter of measuring: a quality model cannot be defined just by its geometric 
accuracy. We have to add parameters that come from the material analysis supported 
by high-resolution photogrammetric data processing: as in the case of rectified images 
and orthophoto to be associated to the 3D models: geometries embodying materials 
and construction techniques contribute to defining enriched 3D model. And, vice versa, 
geometries and interpretation coming from the materials and construction techniques 
can improve the 3D Quality Model. 


#Informative Quality Models (as HBIM, GIS and DBS). Model objects can be pro- 
gressively enriched by the Information, within the informative models; they support dif- 
ferent BIM uses (as the preservation design project, the conservation plans, the materials, 
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decay analysis, construction techniques, structural behavior, Finite Element Analysis, 
energy performances) and decision-making processacross the life cycle management 
(Fig. 5). Parameters define which Information are required to be detected and described 
at the HBIM Information level are fixed by the conservation experts. 


#Mesh Based Surfaces. We have many meshes circulating and coming from automatic 
processes by the users, architects, and engineers, thanks to common tools (as in the 
case of photogrammetric textured models) that could be very useful if used to enhance 
the content interpretation. In general, they are destined to remain at a passive level, 
used as background in the BIM tools. Enabling 3D meshes to embed all the properties 
Information and connections to external DB, as for costs computations, means to turn 
them into Object Models within HBIM. 


#Shifting from Surfaces to HBIM Object Models. We need a shift from the surfaces 
to the object models. Surfaces are the first sentinel that can be easily surveyed, but we 
have to take into account that the surfaces are just a face of solid objects components 
of the architectural heritage or archeological site. We need to go behind the surfaces, to 
understand the Object that has been built and transformed across the centuries with all 
the decay and issues in order to manage quality models and not just the skin surface! 
3D Object shifting means recognizing the *habeas corpus’ of each component as an 
identity card. A wealth of knowledge to valorize the uniqueness of all the architectural 
components: each Object description is a ’unicum’ made by geometries integrated by 
materials and construction techniques. 


#Mapping Materials, Construction Techniques, Decay. Knowledge of materials and 
construction techniques - including the techniques of craftsmen transferred from genera- 
tion to generation across the centuries — are parameters enriching models quality. Material 
analysis, as well as decay analysis, can be mapped generating HBIM Objects to be man- 
aged with external Databases that can be added to the Information by the preservation 
experts and professionals [8]. 3D HBIM quality models can conjugate geometry with 
surface texturing (Fig. 5). Moreover, as demonstrated by many kinds of research, the 
geometry is influenced by the construction techniques as in the vaulted systems where 
the shape is influenced by the arrangements [9]. 


# NDT. Non Destructive Techniques (i.e., IRT Infrared thermal cameras, NIR Near- 
Infrared cameras, Hyperspectral data, geo-radar, sonic analysis). NDT can be useful to 
better understand - behind the surface plaster - the arrangement of different stratigraphic 
units as well as chimney channels [10]. Here applied to the vaults with a passive method 
(a parameter to be considered), thus with fewer results than the rooms’ active heating, 
unfortunately here not possible for the contest. 


#Correlating Material Analysis to the 3D Arrangement. 3D content models capable 
of decoding the complexity of behavior that comes from the stratified transformation 
phases impacted by hazards in a fragile context. 

The out plumbs after the earthquake, the anomalies, the irregularities, the read- 
ing of the different stratigraphic units, the relations among the different components 
(as columns-arches-naves) allowed us to better understand the disconnections and the 
damages preventing disjoining with proper connections in the preservation plan. 
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Fig. 4. The quality of a model depends on the generative model process to get the required scale 
starting from reliable surveying. Material and Decay Mapping as a parameter of quality for HBIM 
data management: NURBS model supports the HBIM Material mapping process. Courtesy of 
ABClab-Glcarus (F. Banfi, C. Stanga, R. Brumana) 
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Fig. 5. Materials, construction techniques, and IRT analysis, together with the decay analysis, 
embedded within the HBIM, supporting the conservation plan Work Breakdown Structure linked 
to the Object models. 3D volume stratigraphy was not integrated into the HBIM. Courtesy of 
ABClab-Glcarus (F. Bafi, D. Oreni, L. Cantini, R. Brumana) 


#Stereotomy. High-quality models digitization is returning to us the history of con- 
structive wisdom and the art of cutting stones in the space (stereotomy): complex shapes 
not belonging to the simplified typological classifications cited in the historical manuals, 
give back us masonry techniques with a multiplicity of results. It is the case of °’ Trompe’ 
shaping (Guarino Guarini) or framed vaults across Europe with the art to build in the 
space complex models saving wood centerings fasting the construction process with a 
unique variety of solutions [11]. 

HBIM models of all the columns embody the out of plumbs together with the 3D 
shaped ashlars elements with the hands-on survey of all the damaged ashlars (Fig.6): 
it was hypothesized an average of 35% of damaged ashlars (yellow) to be necessarily 
substituted during the intervention, in the consciousness that this percentage would 
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undoubtedly increase during the works. But it contributed to hypothesize the ’scuci e 
cuci’ to reduce the replacement to the damaged elements. 


#HBIM Object Libraries. Each single informative object model — considered as a node 
- can contribute to the generation of libraries of the objects across space and centuries; 
these nodes can be compared highlighting permanence and mutations of the construction 
techniques, highlighting the unsuspected richness of the vault systems generating HBIM 
based inventory [12]. Criteria and tools to catalogue brick-masonry vaults through infor- 
mative geographic models contribute to describing the specificity and richness of the 
architectural components [13], fostering the history of historical construction techniques 
[14]. 

To correctly share a model, such as a vault HBIM object, one needs to know: which 
was the commitment purpose and thus the required scale; how it has been surveyed; if 
the model has been generated in a congruent way, that depends on the object level of 
complexity and the type of geometrical survey to generate an Object Model with the 
proper accuracy. Such Information must be included as metadata within the Level of 
Information, of each object model in order to manage the different Levels of Geometry 
within the different phases of development of the preservation process and the correct 
re-use in the data sharing process in the cloud (i.e., throughout Object Libraries). 

The 3D object model of the main façade integrated geometry from laser scanning 
and material documentation by mean of the orthophoto from photogrammetric surveying 
at the GOA20 scale to read DEM morphology after the earthquake and GOAS to read 
the different marble finishing in the different restoration phases in the past connected to 
different skilled workers capacities (Fig. 7). 


#HBIM LOGs-LODs (Level of Geometry and Level of Development): These param- 
eters cross the different phases of the preservation actions and plan by mean of the HBIM, 
as explained in the following paragraph. 


#HBIM WorkBreakdown Structure: HBIM enabled models allows the professionals 
to connect the WBS plan of activities to the information and models, supporting the 
metric and costs computations planning all the activities from the preliminary phases to 
the executive phases. 


4.2 Structural Health Monitoring: #Finite Element Analysis Complex Models 
as a Parameter of Quality (BIM-to-FEA) 


#Structural Health Monitoring (BIM-to-FEA). The importance of taking into account 
all the components within the overall architectural heritage, and their own behaviors, tak- 
ing in account the materials construction techniques, within Structural Health Monitoring 
with their characteristics has been illustrated in many kinds of research: it contributed 
to understanding failures due to long term behavior of heavy structures [15]. 

In the case of the Basilica di Collemaggio the importance of field failures analysis 
of the Basilica S. Maria di Collemaggio has been highlighted [16]. Starting from the 
surveying and on site-analysis, a Finite Element Analysis has been realized using the 
HBIM Objects model (Fig. 8). 
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Fig. 6. HBIM of the octagonal columns, the stereotomic analysis of the ashlars embedded in the 
HBIM, and the restoration with the final result. Courtesy of ABClab-Glcarus (F. Banfi, D. Oreni, 
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Fig. 7. HBIM object libraries: a growing open catalog made by all the richness and specificity 
of the single object model: here the fagade concept, the decorations, and the richness of the Rose 
windows. Courtesy of ABClab-Glcarus (F. Banfi, R. Brumana) 
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The BIM-to-FEA process has been carried out to get the interoperability and model 
transformation for the Finite Element Analysis [17]; the model has been obtained taking 
into account all the structural components as in the case of the HBIM facades objects, 
the vaults, the HBIM octagonal column with the stone ashlar data and arches, together 
with the Information coming from the HBIM-DB of the material and decay analysis 
obtaining the simulations models of the structural behavior. 

It contributed to addressing the identification of the design solution for the replace- 
ment of the crashed dome and roof with the ancient wooden solutions. Different GOAS 
has been adopted in the processing tools by the structural engineering research team. 
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Fig. 8. Structural Health Monitoring: Finite Element Analysis complex models as a parameter of 
quality (HBIM-to-FEA). Courtesy of ABClab-GIcarus (F. Banfi, A. Franchi, R. Brumana) 


4.3 3D Quality Models Enriched by Building Archeology: HBIM 3D Volume 
Stratigraphic Units and Levels of Uncertainties 


#Building Archaeology. Stratigraphic Units detection and 3D volume stratigraphy as 
a parameter of quality models: 3D volume stratigraphy and building archeology taking 
into account the relations among the Stratigraphic Units represent a further parameter 
to describe the quality level of a HBIM, as shown in the Fig. 9 respect the level of 
Fig. 5. The methodological approach of the so-called building archaeology [18, 19] 
helps to clarify not just the recognition of the stratigraphic units [20] starting from the 
material analysis and the decay mapping of the surfaces but also the interdependencies 
and relations among the different Stratigraphic Units (Fig. 10). As an example 3 different 
relationships generally symbolized are described as follows: 
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e Horizontal ‘S’ symbol which hooks two stratigraphic units: Stratigraphic ratio (rela- 
tionship among 2 USs): it binds to, corresponding to contemporaneity and proven 
stratigraphic coherence of two different surface units. 

e Underlined arrow symbol that faces one of the two stratigraphic units: Stratigraphic 
ratio (relationship among 2 USs): leans on..., covers, etc.; symbol applied on the 
part that covers, leans; indicates a ratio of anteriority /posteriority (the surface with 
arrows is rear). 

e Zigzag underlined symbol: Breaks/is broken, cuts/is cut, are an example of options 
applied to the perimeter of the surface that is cut (and therefore pre-existing), and 
that can be identified by the analysis. 


A shift of the 3D quality model toward volume representation represents a further 
level of degree of complexity that the Building archaeology informative modeling turned 
into 3D volume stratigraphy can contribute to decoding [21]. It can be applied to the 
single components, as in the case of facades or vaulted systems [22] to better investigate 
the result of 3D geometry influenced by the construction techniques and/or the transfor- 
mation phases. The Scan-to-BIM process has been extended to Building Archaeology 
informative model to transfer these 3D models augmented by the Information into VR 
experiences [23]. The methodology has been applied to the 3D stratigraphy of the St. 
Francesco Church in Arquata del Tronto components (walls, roof, ceiling, flooring), 
which could be enriched by NDT analysis. Some components of the church (i.e., the 
fagade) were deeper analyzed thanks to Building Archaeology: the Building Archae- 
ology HBIM of the fagade embeds the DB on materials and construction techniques, 
which shows the provenance of the Information in order to make aware of the reliability 
of the collected data that could come from direct (i.e., on-site observation) or indirect 
(i.e., archival document) sources. The material and decay DB related to the HBIM was 
realized according to the Building Archaeology, and each item has its stratigraphic Infor- 
mation, picture, texture drawing, and on-site and historical documentation.. To this aim, 
a Building Archeology property model devoted to managing the properties related to 
the 3D Volume stratigraphy have been defined, including a field where to highlight the 
level of Uncertainty of the relationships among the different transformation phases that 
occurred in the past, given that not all of them have been clarified. 

Informative models can contribute to enriching and feeding VR/AR transferring 
contents to citizens. The case of AR project developed for the St. Francesco church 
represents the potentials of virtual-visual storytelling (VVS) allowing users to convey the 
informative models, including the construction phases, discover tangible and intangible 
values of the building through interactive virtual objects (IVO). 
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Fig. 9. Building Archaeology informative modeling: 3D volume stratigraphy (St. Francesco 
Church, Arquata del Tronto after the 2016 earthquake). Courtesy of ABClabGlIcarus (F. Banfi, 
C.Stanga, R. Brumana) 
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Fig. 10. Toward informative models feeding AR/VR. Courtesy of ABClab-Glcarus (F. Banfi, 
C.Stanga) 


5 Toward Quality Model Specification: BIM to HBIM 
LOGs-LODs, Reversing the Logic Simplex-to-Complex BIM 
Model (Building Construction) in Favor of Complex Modeling 
Knowledge-Creation Since the Design Starting Phases 


#HBIM Level of Developments and Levels of Geometry (LODs and LOGs). Infor- 
mative quality models (as BIM, GIS models) can be gained progressively enriching the 
Information coming from the different BIM uses (materials, decay analysis, construction 
techniques, decorations, surfaces finishing, structural behavior, energy performances): 
they can support the preservation design project and decision-making process till to the 
life cycle management. LOD and LOG applied to HBIM con be considered parameters 
to drive HBIM quality models. 

In the BIM logic of the building construction, the progression of different LOGs 
(Level of Geometry) within the LODs (Level of Development of the different phases 
as the pre-design phase, the design phase till to the facility management) is justified 
by the progressive enrichment of the geometry and Information connected to the new 
construction process (LOG-LOD100-500). When turning toward the preservation aims, 
we have to take into account the specificity requiring since from the starting phase a 
level of richness and complexity in the HBIM modeling to understand the phenomena 
and to carry on the assessment. 
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5.1 # Level of Development Phases Inherited by the BIM Logic and Adapted 
to the HBIM 


BIM LOG/LOD100-200-300-400-500-600 are hereafter adapted to the HBIM specific 
domain of the architectural heritage. The architectural heritage with all its components 
requires the maximum effort of knowledge and details since the starting phase to support 
a decision-making process devoted to proposing sustainable and reliable solutions in the 
design process and to preserve the heritage subject in the intervention and its maintenance 
across the time. 

LODs-LOGs have been turned as follows: 


# LOD100 (Pre Design): the LOD100 (Pre Design) has been turned to data collection 
(LOG100 “Conceptual model, historical reports and archives”); 

# LOD200 (Digital documentation phase): it has been added a “Digital documentation 
phase” (LOD200) to replace the schematic design phase, devoted to the acquisition of 
the “Appropriate geometry (LOG200)” correspondent to the digital 3D survey and on- 
site data collection” not present in the current specification nor in the Heritage domain 
nor in the Infrastructure domain. 

# LOD300 (As-found HBIM model): an “As-found HBIM model” (LOD300) phase 
obtained by the “Precise Geometry, SCAN-to-BIM model object” (LOG300) devoted to 
the HBIM modeling phase has been introduced. As previously described, the adoption 
of one or more scales to model the objects allows professionals to identify the proper 
scale of each component, function of the contest, geometry, funding, and purposes. 

# LOD400 (Design development — Conservation Plan): the Design Phase has been 
shifted and adapted to the HBIM uses, thus addressed to the “Conservation Plan” 
(LOG400). 

# LOD500 (Construction Stage): this phase is simply shifted and tuned to the HBIM 
construction site, supported by LOG500 “Conservation site”. 

# LOD600 (Facility Management): it is integrated by the LOG600 “As-Built, LLCM, 
CDE, HUBs”: it implies, as in the BIM logic, the management, monitoring, and 
communication process from the as-built phase. 


5.2 Levels of Geometry Proposal in the Heritage Domain: HBIM 
LOG100-200-300—400-500-600 


# HBIM Levels of Geometry (LOGs). The proposal to adapt the current LOD-LOG 
definition — so far adopted for the new building domain - has been introduced to manage 
the 3D quality model within the conservation process, taking into account each object 
unicity. The concept of GOAs scales previously defined can be adopted in the function 
of the different phases and needs in the specific LOGs. 

HBIM LODs-LOGs managing the 3Dquality model within the preservation process 
has been introduced not present in the process as in the following cases (Fig. 11). 

The LOG progression is no more related to the scale content defined by the GOA: 
thus, within each development step, the scale can be defined with respect to the user 
needs (Fig. 12). 
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Is it enough to define a Quality Model JUST relying on survey/model precision parameters? 
NOT JUST A MATTER of MEASURING! 
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Fig. 11. The proposal of targeted LODs-LOGs adapting the current BIM LODs-LOGs to the 
HBIM domain. GOAs (models scales) cross all the LOGs, being adopted by the different actors 


involved in the different phases (LOD200-300-400-500-600). Courtesy of ABClabGIcarus (F. 
Banfi, R. Brumana, C.Stanga) 


The HBIM LOGs has been defined as hereafter proposed: 


# LOG100 - Historical report: it has been introduced to collect the historical drawings, 
documents from Libraries, Archives, Maps Collection, Open Data Digital Collections 
(as Europeana), replacing the ‘pre-design’ concept model; 

# LOG200 - Appropriate geometry: 3D surveying and on-site data acquisition have 
been introduced within the HBIM LOGs addressed to the surveying to get 2D/3D drawing 
and mesh digitization, so far totally missed within the BIM logic; 

# LOG300 - Precise geometry: Scan-to-HBIM model Objects generation has been 
specifically introduced to manage the generative HBIM modeling phase to get HBIM 
enabled objects, not supported by the mesh or mass models; 
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BIM - NEW BUILDINGS 


Fig. 12. HBIM turned into preservation aims to support the preservation phases described by 
the Level of Development (LOD) and the Level of Geometry (LOG): the richness of geometry 
obtained since from reliable LOD300 (HBIM object model generation — Precise Geometry) sup- 
ports the LOG400 (HBIM uses) and the Conservation site management (LOG500). Courtesy of 
ABClabGlcarus (F. Banfi, R. Brumana, C.Stanga) 


# LOG 400 - HBIM uses: HBIM-Uses data management (i.e., preservation plan, insert- 
ing the missed material-decay analysis, Work Breakdown Structures- WBS, BIM-to- 
FEA, or Energy Efficiency analysis); LOG400 manages different scale models (GOA 
Scale), complex models with high level of accuracy together with simplified model 
scales. The choice depends on the preservations experts needs but also on tools, as in 
the case of Energy Efficiency tools requiring so far simplified 3D models; 

# LOG500 - Conservation site: HBIM construction site management (CoSiM), on site 
interventions of preservations; 

# LOG600 - Facility Management; it has been addressed to manage Long Life Cycle 
Management (LLCM) starting from the As-Built, within Common Data Environment 
(CDE). 

The richness of reliable quality model geometry obtained since from the starting phase 
by LOD300 (HBIM object model generation) supports the LOG400 (BIM uses), the 
preservation site management (LOG500), as shown in the Fig. 13. 
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5.3 Sharing 3D Quality Models, Re-use and Circulation: Common Data 
Environment Toward XR Platforms and Interoperability 


# Re-use and models circulations. In the UK, Level 2 is also known as Common Data 
Environment (CDE) or a ‘federated model’ where it is possible to use different models 
to create a digital HUB, giving a complete holistic view of the building. 

Assuming that a LOG400 (BIM-uses) can express the concept of model interop- 
erability, it is also useful to consider how this three-dimensional Information can be 
shared among users. LOG500 and 600 introduce the idea of sharing models through 
CDE, increasing the level of collaborative communication: HBIM are evolving into 
interactivity, immersive fruition and interoperability through digital model re-use [23]. 

In the Digital Cultural Heritage (DCH), archaeological sites, architectural her- 
itage, and infrastructures require the management of different Information previously 
described. Consequently, in this specific context, it is necessary to underline that CDE 
should be fed by many data, formats, and models, oriented towards different uses, to 
improve the management of the Objects over time, allowing different users and oper- 
ators to access informative models with additional commitment and limitations in the 
data modifications. 

Therefore, the shared data environment (CDE) should be considered a multi-access 
to multi-source information used to collect, manage, and share graphical models with 
non-graphical data in conventional data formats evolving toward XR platform to support 
AR/VR informative models [24]. According to PAS 1192-2: 2013, a CDE may use a 
project server, an extranet, a file-based retrieval system, or other suitable toolsets. 


‘3D quality informative models’ 
workflow of the different parameters progressively enriching information and models: 
the maximum quality level corresponds to the maximum level of data sharing and re-use 
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Fig. 13. 3D quality informative models possible workflow: managing different parameters and 
progressively enriching information and documents from digitization to XR experiences. Courtesy 
of ABClabGIcarus (F. Banfi, R. Brumana) 


100 R. Brumana 


6 Conclusions and Remarks 


Quality driven models allow a progressively increasing of knowledge creation, opening 
the doors of informed communication using co-working spaces, common data envi- 
ronment, and geospatial hubs with linked informative model objects (as HBIM), in 
the form of Linked Open Data (LOD), enriching and augmenting reality also by using 
VR/AR/MR. Therefore, it requires CDE and platform capable of digesting models pro- 
gressively enriched by the information supporting eXtended Reality experiences. The 
re-use requires to foster common languages for data exchange and vocabularies, tak- 
ing into account the richness and specificity of the construction techniques and of the 
information linked to the HBIM. It also requires filling the gap of lack of skills boosting 
capacity building, especially in the modeling phase growing DG content skills and in 
the use of HBIM by the experts. 

A multi-parameters radar chart application can be implemented to better support the 
users in the decision-making during the 3D quality model generation helping measure 
the level gained during the process. 

Standardization and ad hoc guidelines addressing the complexity of cultural heritage, 
as in the case of LOD-LOG-GOA definition in the HBIM, can contribute to avoiding 
misunderstanding and misuses in the re-use and circulation of the 3D model libraries 
and sites. 
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Abstract. The Antikythera Mechanism is the oldest extant complex geared 
device, and an amazing example of an early analogue computer. It was built 
approximately 2150 years ago. The device was operated manually by a user, who 
would set a date on a dial. All necessary calculations were made using a set of 
gears (at least 39), while the results were displayed on several scientific scales. 
The Mechanism was used to calculate the diurnal and annual motion of the Sun, 
the Moon and probably the planets among the stars. It implemented the astronom- 
ical knowledge of ancient Greeks about the motion of these celestial bodies with 
astonishing accuracy, taking into account the anomalous orbit of the Moon using 
a system of eccentric gears. It could also predict eclipses of the Sun and the Moon 
from the Saros period, which was found in one of its scales. It calculated the dates 
of the major crown games that took place in ancient Greece (e.g. the Olympic 
Games). Finally, it was accompanied by an extended User’s Manual. More than 
20 references to astronomical mechanisms can be found in classical literature from 
50 BC to 500 AD. In this study, the first approach for a holistic documentation of 
the Antikythera Mechanism is presented. 


Keywords: Cultural Heritage - Holistic documentation - Antikythera 
Mechanism - Gears - Ancient Astronomy - Ancient Technology 


1 Introduction 


The Antikythera Mechanism is a highly complex object, and since it was found in 1900 
has been the subject of many extensive studies. It is one of the oldest extant geared devices 
and was an astronomical “computer” used for the calculation of the diurnal and annual 
motion of the Sun, the Moon and potentially the planets, as well as for calculating the 
dates of the Olympic Games. From the extensive studies [1—4], we have a huge amount of 
data and as such this is an excellent case study for proposing an approach for the holistic 
documentation of cultural heritage objects. This paper is made up of two constituent parts, 
the first outlines an approach for how the Mechanism can be holistically documented 
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within a conceptual data model, and the second outlines some of the data that is necessary 
to be collected and recorded in order to holistically document the asset. Both of these 
sections have data representational of what is available, from the physical object and 
the relevant contextual information (modern, ancient, astronomical, geographical). An 
integral part of this being classed as a success story comes from ensuring that both the 
digitization process (i.e. the use of x-ray tomography and Polynomial Texture Mapping 
(PTM) Dome technology) and the documentation process, including how the data is 
represented and mapped in a digital format, are done to a high quality by engaging with 
the complexity of all of these factors. 


2 Identifying Necessary Data for the Holistic Documentation 
of Movable Archaeological Findings 


Cultural Heritage (CH) objects are distinguished between tangible and intangible as well 
as movable or immovable. One of the most iconic and complicated museum object is 
the Antikythera Mechanism. The Mechanism proves that the Greek engineers of the 
Hellenistic period were far more advanced in the design and manufacture of geared 
devices than the surviving written sources imply. 

The digital holistic documentation of movable objects, like the Antikythera Mech- 
anism must include: all possible data regarding the physical object, including the 3D 
geometry, the materials of its construction, the ancient context including intangible ele- 
ments such as, its purpose, utility, handling and operation, its geographical and chrono- 
logical origin and the modern excavation and analysis data. Based on the information 
gathered, the identification of the scientific and technological knowledge necessary for 
the design and construction of such a complex device should be recorded. Simultane- 
ously, a review and documentation of the existing knowledge regarding the mechanism 
during the period of its construction should be completed. Furthermore, accurate digital 
and physical representations of the original discovery must be created, as well as, the 
complete reconstructed functional copies of the object. 

In the following paragraphs a representation of all the data necessary for a holis- 
tic documentation of the Mechanism are presented. The more detailed explanation of 
this data and the subsequent discoveries regarding the Mechanism will be presented in 
Sect. 3. These data are available in digital form and can be easily introduced into an 
integrated digital environment with corresponding interfaces, so as to create a digital 
holistic recording of the Mechanism. 

A conceptual model of data for the holistic documentation of the Antikythera Mech- 
anism was created, dividing up into the categories of different types of data needed in 
order to holistically record a CH object. The initial stages of the model, from “Tangible” 
through to “Mechanism” follow the general taxonomic system under construction in 
the MNEMOSYNE project [30], and categorises the Mechanism within the classes of 
“Movable”, “Instruments and Manufacture”, “Astronomy” and “Mechanism” which indi- 
cates the type of the object (Fig. 1). From this point we start to represent the data needed 
to record each tangible facet (“Elements”) of the physical object and its intangible infor- 
mation (“Context”) — both historical and modern - which will be explained further in 
Sect. 3. 
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Within this conceptualization, the data is separated out into contextual information, 
which includes information relating to the “Ancient” and “Modern” contexts — including, 
its production, use, deposition, excavation and post excavation studies, as well as relating 
information regarding the “Geographical” and “Astronomical” data, which provides 
important explanations and knowledge, that enhance our understanding regarding the 
need for and the use of such an object. The other main category is “Fragments”, which 
refers to the information related to the physical object. It is important to highlight, that 
the Antikythera Mechanism is a particularly complicated object; many elements of the 
structure have been proposed but no physical remnants have been found — for example 
some of the gears, much of the frame etc. Additionally, the mechanism has been found 
in a calcified fragmentary form with large portions remaining undiscovered, thus, many 
of the links between elements have been proposed through extensive studies using the 
application of high level technologies such as x-ray tomography and model building. 


Tangible 


Movable 


Manufacture 


Astronomy 


Mechanism 


Context Elements 


Ancient Modern Astronomical Geographical 
Frame Plates Dials Pointers Shaft Gears Pin-and-slot 


Research Excavation Sun Moon Zodiac Eclipses 
Date Location 
Form Function Material Technique State and 
Condition 


Studies Site Date Person(s) Solar Lunar 


Teams Themes 


Fig. 1. Conceptualisation of “holistic” documentation regarding the Antikythera Mechanism. 


The collected information was carefully consolidated to support contextual knowl- 
edge redistribution and preventive conservation of historic assets through heritage signif- 
icance awareness. Holistic recording of the context of the mechanism revealed a series of 
important categories (Fig. 2). The categories of data types are connected by solid lines, 
whilst values and object related data are represented by dotted lines, to differentiate 
between the different aspects of the model (the categories versus the value based data). 
Firstly, the modern context including the excavation, related objects from the wreck, 
conservation, storage and related research projects related to the mechanism. Stages of 
data acquisition and processing leading to the production of digital and physical recon- 
structions of the mechanism are shown in Fig. 7. The figure highlights the sheer quantity 
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and variety of data (and data formats) involved in the digitization process of documenting 
such a complex object; from cutting-edge technologies of x-ray tomography, through 
to the necessary manipulation of the multi-layered data sets handled within sufficiently 
detailed point clouds. Secondly, the ancient context, including the manufacture, use and 
deposition. Thirdly, the astronomical context, revolving around lunar and solar occur- 
rences and astronomical knowledge that was woven into the manufacture and use of the 
object. Finally, the geographical context, which is also integral for understanding the 
ancient context of the object. The mechanism was calibrated to work optimally between 
33.3-37.0° latitude, which falls in the mid-Mediterranean range, and is consistent with 
Rhodes; a possible manufacturing site [16]. Recording and studying related information 
from a broad range of areas (from physical objects from the same context, to astronom- 
ical research), have been integral for piecing together the knowledge regarding and the 
story of the object, so recording all of this related information within the same system is 
extremely important. Figure 2 shows the overall taxonomy of the context (i.e., the intan- 
gible information). Figures 3, 4, 5 and 6 present the detailed taxonomy of the branches 
associated with the context, which show the various data and metadata individually 
related to the discovery, research and history of the asset, as well as its astronomical uses 
and geographical information. 

Figure 8 shows the overall taxonomy of the elements (i.e. the tangible information) 
that compose the mechanism. Figures 9, 10, 11 and 12 present the detailed taxonomy of 
the branches associated with these elements, which show the various data and metadata 
individually related to the frame, dials, pointers, shafts and gears. The elements in blue 
have been physically found (or identified within tomography images) and the elements 
in pink have been proposed by the various research teams, but have not been physically 
found. 

The main criteria that are modelled here, or are expected to be modelled within the 
next period are: 


e Form - referring to the physical form of the object (i.e. the number of teeth on the 
gears), 

e Material — all data relating to the material and chemical composition of the object, 
considering the impact of the seawater and the conservation process on each element. 

e Function -referring to the purpose and use of the element. An additional consideration 
here, taking into account the complexity of the object, is how each of the elements 
(existing or proposed) connect together and their combined output. 

e Technique — ideally this would refer to each aspect of the “technique” aspects, this is 
a term that can be applied as an addendum to function, as in the technique for which 
the elements work together to perform an action, but can also be a term to cover the 
methods of how each element was manufactured and the mechanism constructed. 
State/Condition — data relating to the preservation and conservation of the element. 
Location/Context — data relating to where it was found/alongside what, this maybe 
used to indicate which, in this case, elements connected/functioned together, as well 
as in which fragment they were found. 


It is important to clarify that each of these criteria, whilst initially modelled inde- 
pendently, will be interlinked where necessary and they will be further enriched with the 
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Fig. 2. Conceptualisation of data regarding the contextual information of the Antikythera 
Mechanism. The expansion of these branches is shown in Fig. 3, 4, 5 and 6. 


contextual information. It should be highlighted that the mechanism consists of many 
small parts of equal importance working in synchronization. 

As can be seen in the example of Antikythera Mechanism, within the category of 
elements, each constituent part is separated out into form type, i.e. gear, pointer, plates, 
dials, shaft, pin and slot mechanism and frame and this is then subdivided into the 
actual or proposed element, i.e. a specific gear. From this stage, the data represented 
is extracted from the various studies outlined throughout Sect. 3 of this paper, some of 
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which is contradictory or at least come to different conclusions, which is often the case 
among different study groups and technological advancement. A key example of this is 
the number of teeth on a gear, which have been postulated upon by a number of different 
researchers. However, this variable research is part of the holistic process and feeds into 
and supports the holistic documentation of the object. 

Empirical evidence suggests that digitization concerns have overgrown technical 
issues of conversion and often expand to cover the management of collections, medi- 
ation and representation of cultural heritage in the digital environment, economics of 
digital repositories, business models, quality and sustainability of digitization initia- 
tives etc. This type of interrelating categorizations was meant to form an all-inclusive 
functional data model, consistent with role-based online analytical processing (OLAP) 
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operations. This means they could easily serve as a benchmark for integrated visual- 
izations built in powerful dashboard design software packages that come with visual 
analytics capabilities; thus, enable specific users to conduct ad hoc queries (and filter 
on-demand), with little or no programming skill. One very interesting application is the 
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ability to recursively traverse a dimension hierarchy and print-out aggregations at the 
actual browsed location. For example, providing grounds for an engineering specialist to 
select individual shafts and inspect the corresponding engaged gears (Figs. 13 and 14). 
As can be seen in these figures, the items listed in the hierarchy (left-hand side) are taken 
directly from the elements of the asset’s taxonomy, which are shown on the right-hand 
side of Figs. 13 and 14. The selected individual shafts and their corresponding engaged 
gears, coloured in blue, are visually distinguishable. 
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Fig. 14. Antikythera Mechanism. Shaft “b” and gears “b01”, “b02”, “b1”, “b2”, “b3”. 


3 The Existing Data for the Holistic Documentation 
of the Antikythera Mechanism 


3.1 The Antikythera Shipwreck and the Discovery of the Mechanism 


In 75 BC a large Roman ship sailed between the mainland of Greece and Crete. The boat 
sank off the shores off the small Greek Island Antikythera. The ship was loaded with 
works of art and other precious artifacts. Two thousand years later, during Easter in 1900 
AD sponge-divers from the Greek Island of Symi, accidentally discovered the ancient 
shipwreck off the coast of Antikythera. The underwater excavation began at the end of 
November 1900 and a few months later important findings were recovered, such as the 
famous Antikythera Ephebe [4] (Fig. 15). Most of these findings are now exhibited at 
the National Archaeological Museum of Athens [4]. Coins from Pergamon were also 
retrieved, which gave the wreck a relative date between 86 and 67 BC [5]. 


Fig. 15. The Antikythera Shipwreck and the underwater excavation. 


Amongst the findings, a strange bulk of material that was broken, worn and calcified, 
with signs of the presence of bronze was found (Fig. 16) [4]. In the first publication of 
the Antikythera shipwreck [6] the existence of the Mechanism was mentioned with the 
suggestion that it was an astronomical instrument. Through investigation, the Antikythera 
Mechanism, had the potential to change the knowledge we had so far on the technological 
skills of ancient Greeks. 
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Fig. 16. The biggest fragments of the Antikythera Mechanism [4]. 


3.2 The Investigation of the Antikythera Mechanism 


The first scholar, who studied the function of the Mechanism extensively, was Derek de 
Solla-Price, with the help of Charalambos Karakalos from the Research Centre Demokri- 
tos in Athens. He worked for more than 30 years and eventually published an extensive 
account, “Gears from the Greeks” [7]. He declared that “the Antikythera Mechanism is 
the oldest proof of scientific technology that survives today and completely changes our 
view of ancient Greek Technology”. 

After de Solla-Price, research was undertaken by Michael Wright and Alan Bromley. 
Unfortunately, Alan Bromley died in 2002. However, Michael Wright published a series 
of papers, where he postulated that the back dials of the Mechanism were spirals and 
that the upper dial was built to follow the draconic lunar month. He also elaborated on 
the pin and slot mechanism (see Sect. 3.2.1.1) and proved its epicyclic function [8]. 
Additionally, he made strides towards creating a reconstruction of the Mechanism and 
he produced superb bronze replicas. 

In 2001, Mike Edmunds and Tony Freeth (Cardiff University), Xenophon Moussas 
and Yanis Bitsakis (University of Athens) and John Seiradakis (University of Thessa- 
loniki) created the “Antikythera Mechanism Research Group”. They received a grant 
from the Leverhulme Foundation, U.K. and the permission to undertake a new investi- 
gation from the Ministry of Culture of Greece. After the permission was granted, Eleni 
Magkou and Mary Zafeiropoulou (National Archaeological Museum) and Agamemnon 
Tselikas (Cultural Foundation of the National Bank of Greece) joined the team, which 
was soon supported by an international team of astronomers, archaeologists, mathemati- 
cians, physicists, chemists, computer engineers, mechanical engineers, epigraphologists 
and papyrologists. 

In September 2005, they undertook a major new investigation of the Antikythera 
Mechanism, using an innovative and state of the art high power micro-focusing x-ray 
tomography, specially constructed by X-Tek Systems [9] (Fig. 17, left) and the Hewlett 
Packard, USA, PTM Dome technique [10] (Fig. 17, right). In November 2006 the results 
of the investigation were announced during an international conference in Athens and 
published in the international journal Nature [11]. This technique allowed the acquisition 
of three-dimensional images of the fragments of the ancient mechanism. The images 
were examined to reveal internal details of gearing and inscriptions that had hidden due 
to the preservation state of the fragments which remained underwater for more than 
2000 years and the previous lack of the necessary technology to access this information. 


116 K. Efstathiou et al. 


All inscriptions are written in Greek. A new font (True type fonts) was developed at the 
Aristotle University of Thessaloniki, in order to reproduce the fine art letters. 

Realizing the importance of the Antikythera Mechanism, one of the largest active 
groups of researchers was formed in the Aristotle University of Thessaloniki (AUTH) 
of Greece, which they have worked to produce numerous publications on the analysis 
and reconstruction of the mechanism [12-16]. 


Fig. 17 X-ray tomography (left) and PTM Dome technique (right) applied for the study of the 
Antikythera Mechanism. 


3.2.1 Investigation Results 


3.2.1.1 Description of the Antikythera Mechanism [2-4, 11, 12, 14-16] 

The Antikythera Mechanism is relatively small, approximately 30 cm x 20cm x 10cm 
in size, which was found in poor preservation state, consisting of 82 fragments that have 
been studied together to reveal the information and data that will be discussed within 
the following sections (Fig. 18). The largest seven fragments are named with the letters 
A, to G and the smallest fragments are referred to as numbers 1-75. 


Construction’s Materials of the Mechanism 

The material used to reconstruct the various parts of the Mechanism, except for its 
wooden mounting box, is bronze, a copper - tin alloy. In the period from 1970 to 1974, 
chemical analysis was performed to determine the chemical composition of the metal 
alloy used in the manufacture of the Mechanism. The chemical analysis was performed 
by a spectroscopic method. Two small fragments of the Mechanism were studied. The 
chemical analysis showed that the fragments were made of bronze, with a tin content of 
about 5% [4]. Newer analyzes by Panagiotis Mitropoulos in 2018 revealed three alloys, 
the main components of which are copper, tin and lead [4]. The shares of copper tin 
and lead varied. It can be assumed that the individual parts of the mechanism consist of 
copper alloys of different composition [25]. 

Structural Elements 


Gears 

The gears found in the Antikythera Mechanism are the earliest known to resemble the 
shape and design of modern gears. Their triangular teeth were designed to transmit 
angular motion, not power. Detailed studies of the fragments of the Mechanism revealed 
that it had at least 39 gears, 29 of which have been identified in the largest of the calcified 
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Fig. 18. Fragments of the Antikythera Mechanism from the National Archaeological Museum of 
Athens. 


fragments - 28 in Fragment A and | in Fragment C. The existence of ten more gears 
has been determined taking into account astronomical calculations [11, 12, 20, 21]. A 
functional diagram of the gear trains is presented in Fig. 19. The gear pairs are displayed 
with the driving gear first (to the left), except for the triple where gear b2 drives the two 
other gears, cı and |). 

The Antikythera Mechanism incorporated also 2 crown gears as well as 19 shafts 
and axles with complex geometry. Two pairs of them are concentric, which means that 
one of the two shafts passes through the other. The two shafts are rotating independently. 
Moreover, there is an axis, which has two eccentric cylindrical bearing points for two 
gears. At various locations, mainly inside the Mechanism, there were also various com- 
ponents that supported the shafts and other parts of the device to make the construction 
robust and functional. 


Pin-and-Slot Mechanism 

In the pin-and-slot mechanism (Fig. 20), the axial distance of the rotation’s axis of the 
two gears is to approximately 1.1 mm. The lower gear has a pin that engages with a 
slot on the upper gear, forcing it to rotate. The epicyclical movement of the upper gear 
tracked the motion of the Moon in the sky with great accuracy [18, 19]. 


Dials and Inscriptions 

The Antikythera Mechanism was a complicated instrument. By using x-ray micro- 
focusing tomography, inscriptions that have not been read for more than 2000 years 
were revealed. Approximately 3500 letters and symbols have been deciphered so far. 
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Fig. 19. Gear trains of the Antikythera Mechanism. 
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Fig. 20. The pin-and-slot mechanism [20]. 


There meaning fall into three broad categories: astronomical inscriptions, geographi- 
cal inscriptions and technical inscriptions. Several astronomical terms have been read 
referring to the Sun, the Moon, the ecliptic, the Metonic and Saros cycles and other 
astronomical phenomena. The word “X)THPIT MO” (stationary point) is mentioned 
several times, likely referring to planetary stationary points. 

The Mechanism had 7 pointers, which provided 8 indications in the scales of the 
Mechanism (the pointer of the Moon gives two indications). It had three main dials, one 
at the front side with two concentric scales, and two at the back in the form of spirals. 
On the front side (Fig. 21, left), there were two concentric circular scales. The outer 
scale had 365 subdivisions the days of the year. The inner scale had 360 subdivisions 
and the names of the 12 zodiac constellations. The date scale had the shape of a ring 
(Fig. 21) which was removable. Behind this date ring, there were 365 holes, and the ring 
was secured at one of the holes using a pin. Every four years, the operator was able to 
remove the ring and move it by one hole, to take leap years into account. 

A very careful analysis of the gears’ co-action revealed their use in calculating to 
a high level of accuracy, the position of the Sun and the Moon on the zodiac (the zone 
that contains the zodiac constellations). This position was shown by the pointers of the 
front dials. Similarly, a crown gear drove a black and white coloured spherule, showing 
the current phase of the Moon. 

On the back side of the Mechanism (Fig. 21, right) there were two spiral scales. The 
upper spiral consists of 5 windings, with its total length divided in 235 sections. Using 
this dial, the user could read the position of the Moon within the Metonic cycle of 19 
tropical years, which is almost equal to 235 synodic (lunar) months. The Metonic cycle 
is almost equal to the least common multiple of the tropical year (365.2422 days) and the 
synodic month (29.5306 days). The difference between the two periods (of 19 tropical 
years and 235 synodic months) is only 2 h. This knowledge allowed the calculation of 
the exact day a full Moon would occur, which is a very useful knowledge for agricultural 
or nautical activities 2000 years ago, when no electricity was available. The accuracy 
of the position of the Moon was achieved by a pin-and-slot mechanism [8, 17] that 
reconstructed Hipparchus’ first anomaly of the Moon’s motion (due to its elliptical orbit 
around the Earth). 

This anomaly is, in fact, Kepler’s 2™ law. 
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Fig. 21. The front side with the removable date scale ring and the back side. 


Calippos 100 years later corrected Methons calendric system. Every 4 periods of 
Meton, i.e., every 76 years, one day needed to be removed. The Callippic pointer of the 
subsidiary dial within the upper back spiral of the Antikythera Mechanism indicated 
when the correction must take place. 

A subsidiary dial within the upper back spiral of the Antikythera Mechanism dis- 
played the celebration date of the ancient Panhellenic crown games. On the circumference 
of the dial the words Olympia, Pythia, Isthmia, Nemea and Naa have been deciphered. 
In each quadrant of the interior, the four years Olympic cycle are indicated. All these 
games were crown games, with winners being rewarded with olive branch crowns. 

The lower back dial is a Saros eclipse-prediction dial, arranged as a four-turn spiral. 
This dial contained the 223-month eclipse Saros cycle (of approximately 6585.3213 days, 
or nearly 18 years and 11 1/3 days). 223 lunar months (one Saros cycle) after an eclipse, 
the Sun, Earth, and Moon return to approximately the same relative geometry, and a 
new, nearly identical, eclipse cycle begins. The Saros cycle was marked with the dates 
(month, day, and hour) when a possible solar or lunar eclipse would occur. The markings 
were engraved with symbols (“H” — HAIO® — Sun and “X” — SEAHNH — Moon, etc.), 
The fact that both letters, “H” and “£”, appear simultaneously in some glyphs, most 
probably means that the glyphs represent predictions of future eclipses and not records 
of past eclipses. 

A subsidiary Exeligmos dial, within the Saros dial, extended the eclipse prediction 
capabilities to three Saros cycles, indicating that 8 and 16 h should be added respectively 
in the second and third Saros cycles to the eclipse times indicated by the inscriptions. 

The most impressive part of the Mechanism is related to the moon phases and the 
moon’s movement anomaly. In the seventeenth century, Johannes Kepler, in relation to 
the sun and moon movements, claimed that the “holy circular motion was not circular" 
and suggested that it was an ellipse orbit (first and second law of Kepler). Inside the 
Mechanism, a gear system is identified, that simulates this motion with a high-level of 
accuracy. 
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The display of this movement was achieved by the use of two eccentric gears (Fig. 20), 
taking into account the moon’s movement anomaly caused by its eccentric orbit around 
the Earth, was achieved by the use of two eccentric gears (Fig. 20). 


The Fragment D of the Mechanism 

The latest outcomes of the AUTH team are related to the investigation of fragment D. 
In all modern studies of the Mechanism, fragment D is considered as a lone fragment 
and does not exist in any constructed model of the Mechanism. After a long-term study, 
it was determined that Fragment D is indeed part of the mechanical arrangement that 
calculates the equation of time [2]. 


3.2.1.2 The Antikythera Mechanism, from Physical to Digital 3D Representations 
Starting in 1998 by M. Roumeliotis [26], to date many animations and simulations of 
the Mechanism have been created, which can be searched online. 

Regarding physical models, from 1900 until today, many scientists have involved 
with a reconstruction of the Mechanism. In 1928 Admiral I. Theofanidis with the contri- 
bution of E. Zinner, R.T. Gunther and W. Hartner, listed the visible gears and circles from 
the back side of the Mechanism and they characterized the Mechanism as an astrolabe. 
Admiral Theofanidis achieved to read 350 letters of the inscriptions and was the first 
who tried to make a replica model of the Mechanism [4, 20]. 

De Solla Price studied the Mechanism from 1950 until 1974. First, he analyzed the 
constructed materials. Under the supervision of De Solla Price, Robert J. Deroski was 
constructed in 1975 two replicas of the Mechanism as a proposal for the possible function 
of the device [4, 20]. Afterwards various manufacturers reproduced Price’s models with 
few functional deviations, such as the models of J. Gleave in 2000 and D. Kriaris in 1999 
and 2007 [4, 20]. 

In 1985, A. Bromley has continued the research of Price including his remarks. At 
his first attempt he cooperated with F. Percival and a replica of the Mechanism was 
constructed Five years later in 1990, he cooperated with M. Wright, and they used x- 
ray Linear Motion Tomography in order to define the array of the gears into space. 
This method didn’t achieve the desired results. M. Wright claimed that the Mechanism 
could predict the movements of the Sun, the Moon and of five planets [19]. In 2005, he 
constructed a model using approximately 40 gears [8]. 

In 2008, the Aristotle’s University of Thessaloniki began to develop models of the 
ancient Mechanism [11-13]. In 2011 five accurate and functional models were con- 
structed (Fig. 22). From then until now many updated accurate and functional replicas 
in scale 1:1 as well as in scale 3:1 are constructed and are exhibit in various museums 
and other institutions worldwide [20, 28, 29]. 

The AUTH used complicated digitization processes, such as, using three - dimen- 
sional images obtained from the x-ray tomography, the PTM Dome techniques and using 
the Volume Graphics specific software system VGSTUDIO MAX [22], to develop 3D 
digital models of the four basic fragments (Fig. 16). Using the same data, the whole 
Mechanism was digitally recreated. In Fig. 23 and Fig. 24 illustrates the 3D digital 
reconstructions of the gear trains and the whole functional Mechanism is presented. 
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Fig. 23. 3D digital reconstruction of the Mechanism [20]. 


Figure 24 illustrates how the front side of the Mechanism (left) and the gear system 
(right) was designed based on components revealed from the x-ray tomography images. 


Fig. 24. Mechanical parts localized in the tomographies and corresponding designed elements. 


3.2.1.3 Importance, Purpose, Utility, Operation, and Handling of the Mecha- 
nism 
3.2.1.3.1 Importance and Purpose of the Mechanism 


Taking into account the theoretical and technological knowledge required for the con- 
struction of the Mechanism, it can easily be ranked amongst the Wonders of Ancient 


World [2]. 
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The Antikythera Mechanism was the first surviving geared analogue computer in 
history. The next extant geared device is a Byzantine clock calendar, which was built 
in the 5" or sixth century, and the next mechanical calculators were built more than 
800 years later. Other examples of a similar complexity to the Antikythera Mechanism 
came much later, with at the beginning of the thirteenth century the astronomical indicator 
of Wallingford, 50 years later (1348-1364) the astronomical clock of Dondi, and in 
1410 the Prague astronomical clock were developed. Later in the seventeenth century, 
the calculator of Schickard (Kepler’s collaborator) and the Pascaline of the great French 
scientist Pascal were built. The Antikythera Mechanism inspires manufacturers to date 
as for example the design of the years dial of the Atmos millennium clock [23] and the 
mechanical computer of the spacecraft “Automaton Rover” [24]. 


3.2.1.3.2 Utility, Operation, Handling and prediction’s Accuracy 

The main use of the Antikythera Mechanism was to calculate the exact position of the 
Sun, the Moon, the phases of the Moon and the lunar or solar eclipses and possibly the 
position of the planets in the sky. Additionally, besides the predictions of astronomical 
events, the Mechanism could determine dates related to religious, social and agricultural 
rituals and events. 

As previously discussed the subsidiary dial on the upper back spiral of the 
Antikythera Mechanism displayed the dates of the Olympic Games, which were held 
during the first full Moon after the summer solstice. The Mechanism could accurately cal- 
culate this date as well as the date for the Panhellenic crown games of Isthmia (Corinth), 
Nemea (Nemea), Pythia (Delphi), Naa (Dodona) (Fig. 25) [21]. 


Fig. 25. The subsidiary dial displayed the dates of the Panhellenic crown games. 


In Fig. 26 the Parapegma of the Mechanism is shown [16]. The Parapegmas were 
essentially calendars of astronomical and meteorological events, which were widely used 
in ancient Greece. The astronomical events referred in the Parapegmas are events that 
associate the sunrise and sunset of stars or constellations with the sunrise or sunset. For 
many years, seasons, with their different climatic conditions, were an important unit of 
time as they played a crucial role in people’s lives. Over time, however, it was found that 
the start date of each season, could not be determined with the usage of classic calendars, 
based on lunar months. Therefore, people turned to stable natural phenomena, to define 
the seasons. Some of these phenomena were the rising or sinking of some stars. These 
phenomena appear every year at a fixed date in the sky. The occurrence of these events 
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once during a solar year has contributed to use them in order to organize practical-social 
activities such as agriculture and navigation [16]. Hesiod mentions that the harvesting 
period is the season when the constellation of Pleiades appears for the first time in 
the sky, and the time of plowing is the time after the temporary disappearance of the 
Pleiades. The grape harvest must take place when Arcturus appears in the sky for the 
first time. Such phenomena and the date on which they will occur, are predicted using 
the Parapegma on the Mechanism [16]. 


EIEXREPIA | 
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Fig. 26. The Parapegma of the Mechanism. 


The rotation of any of the gears or any of the pointers gives movement simultaneously 
to all other gears and subsequently cases the seven pointers to move, which shows the 
various astronomical phenomena in the related mathematical scales. Based on an analysis 
of the applied torque of the operator when handling the Mechanism, and the necessity 
of a precise positioning of the pointers, e.g. the Sun/Date pointer on the front of the 
Mechanism, the most probable scenario for the handling of the Antikythera Mechanism 
is the rotation of the Moon pointer [2]. The operator, by rotating the moon pointer and 
choosing a date, can determine the astronomical phenomena that may occur on that day. 
Respectively, by choosing an astronomical phenomenon it can be observed on which 
date it will happen. 

Regarding the accuracy of the predition, it is mentioned that, from the construction of 
the Mechanism to date, the position of the constellations in the sky shifted approximately 
30°, which corresponds to one zodiac constellation. To find out the accuracy of the 
Mechanism’s predictions for the present era, in the model shown in Fig. 27 we have 
captured on the front of the Mechanism, the current positions on the scales of dates and 
zodiacs. On the front side of the Mechanism, the indications of the pointers of the sun 
and the moon during the operation of the Mechanism, for the 15 June 2011 are shown. 
Based on current estimates for June 15, 2011, the predictions were: Sun in the Taurus 
Constellation, Moon in the Sagittarius Constellation, Full Moon, and a Lunar eclipse at 
23:00. The corresponding displays on the front side of the Mechanism for this date are 
exactly the same (see Fig. 27): Sun in Taurus, Moon in Sagittarius Constellation and 
Full Moon. Thus, a moon eclipse was possible. 

Observing the pointer of the eclipses on the back side of the Mechanism for this date 
(Fig. 28), a lunar eclipse would likely occur. 

Regarding the moon eclipse, the Mechanism showed: a) Eclipses pointer: £ (Xernvy 
= Moon), © = 9 and b) Exeligmos pointer: H = 8. 
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Fig. 27. Displays on the front site of the Mechanism for June 15, 2011. 


Exeligmos Dial Exeligmos pointer 


The first hour in 
the antique was the 6" clock 
morning of our time. 
Eclipse pointer: © = 9 
Exeligmos - Pointer: H = 8 
6+9+8=23 
g The full cover of the moon 
through the shadow of the earth 
will take place at 23:00 


2.: 2eAnvn = Moon Eclipse 


5 Z | — £ : Opa = Time 
Prediction of eclipses È o ie 
(Saros Dial) © :=9t Hour 


Fig. 28. Predictions on the lower part of the back side for June 15, 2011. 


Taking into consideration, that the first hour in antiquity was the 6" hour the day, 
the full coverage of the moon from the shadow of the Earth will occur at 23:00 (9 + 8 
+ 6), exactly at the same time of the prediction of the device. 


3.2.1.4 Place and Date of the Construction of the Antikythera Mechanism 

The Mechanism was possibly built in Rhodes, between 150 and 200 BC. Early evidence 
of similar machines, are references reported to Archimedes (287-212 BC) as a construc- 
tor of devices which depicted the celestial bodies [2, 3]. The Mechanism cannot have 


126 K. Efstathiou et al. 


been built later than the shipwreck dated by the Pergamon coins within a few years of 
60 BC. The inscriptions letter style suggested that it was constructed in the period of 
50-200 BC. An important date could be set if we knew when the parameters required 
for the pin- and-slot lunar anomaly Mechanism were first deduced. It is known through 
Ptolemy that Hipparchos did characterize and quantify the anomaly by epicyclic and 
eccentric models of the lunar orbit. Therefore, it would be necessary for Hipparchos’s 
values to be involved the manufacture so it must have taken place after 170 BC [2]. 

The optimum latitude for fitting the astronomical phenomena listed in the parapegma 
of the Mechanism is consistent with the mid-Mediterranean around 35° [16]. Rhodes 
(36°) remains as the most likely candidate. The Antikythera ship may have called there 
before it wrecked. Rhodes was known as a highly technological naval port with a thriving 
bronze industry. It was home to Hipparcho and is the place for which we have a record 
of the Mechanism being sighted. It may also explain the presence of the Halieia Games 
on the Games’ dial as they were held in Rhodes [2]. 

Another connection to Rhodes is the construction of similar Mechanisms, which are 
referenced by Cicero, from when he visited the laboratory of Poseidonios (135-51 BC) 
in Rhodes, where he admired a celestial sphere made by Poseidonios [2]. 

The Metonic cycle, found in the upper back dial, contained a full calendar, which 
is repeated 19 times. Comparing this calendar with the calendars of the ancient Greek 
cities, it was found that it coincided with the cities of Kerkyra, Vouthrotos and Dodona (in 
NW Greece) and Tavromenion (in Sicily) [16]. This could indicate that the Antikythera 
Mechanism was used, but not, necessarily, constructed in NW Greece. 


3.2.2 Documentation of the Necessary Knowledge for the Mechanism Design, 
Construction and Manufacturing at the Time and Place of Its Construction 


3.2.2.1 Scientific and Technological Knowledge Necessary for the Design 
of the Mechanism 

A question, which arises, is whether the Greek astronomers 2200 years ago, had the nec- 
essary knowledge to calculate astronomical phenomena, using the Mechanism. Various 
sources, discussed below, show that they did. 

The only accurate clock that the Ancients had was the moon months. Meton in 
the fifth century BC, connected the moon calendar to the annual. He calculated that 
19 years include 235 lunar months [2]. Callipos (370-300 BC) calculated that Meton 
was making a mistake one day every 76 years [2]. These calculations are performed 
by the Mechanism and appear on two scales on the back side of the Mechanism. The 
Callippic pointer of the subsidiary dial within the Meton scale (upper back spiral of the 
Mechanism) indicated when the correction must take place. 

Aristarchus (310-230 BC) was the first astronomer in history, who discussed the 
heliocentric system [20], he calculated the size of Earth, Sun and Moon. Hipparchus 
[20] (190-120 BC) is considered as one of the greatest astronomers of all time. He 
lived contemporaneously to the time of the construction of the Mechanism in Rhodes. 
He calculated, the transient motion of the Earth which lasts 25,800 years. Another 
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calculation developed was the determination of the distance between the Earth and the 
Sun & the Earth and the Moon. 


3.2.2.2 Technological Knowledge Necessary for the Manufacturing of the Mecha- 
nism 

Another question, which arises, is whether the ancient Greek mechanicians had the 
necessary technological knowledge for the manufacturing of the Mechanism. Various 
sources show that they had them. 

Until the discovery of the Antikythera Mechanism the construction of the first real 
gears was dated centuries later. Aristotle describes the rotation of cylindrical objects 
by friction, due to the roughness of their cylindrical surfaces. The creation of surfaces 
with higher roughness slowly led to the development of teeth and gears. The gears of 
the Antikythera Mechanism (second century BC) are the first known example. Two ref- 
erences related to calculations and constructions of gears are from Heron and Pappus. 
Heron of Alexandria was an engineer and geometry, who lived in Alexandria, Egypt. 
In his description of the construction of an odometer he mentions and describes gears. 
Pappus of Alexandria describes several machines that were described by earlier mathe- 
maticians and engineers, such as Archimedes, Heron, etc. and included gears [26]. For 
two gears to work together, they must have the same ratios of diameter to the number 
of teeth. This relationship is called today “module”. From Papus’s writings, it is clear, 
without any doubt that the Greeks knew the module in antiquity. 

Very likely, the gears of the Mechanism were made of cold forged thin bronze plates 
by sawing, removing redundant material and leveling with a hammer [25]. 

In order to manufacture the mechanism particular tools were necessary. The text of 
the inscription from the fourth century BC shown in Fig. 29 concerns the construction 
of bronze axes “II6X0c” for the Filonian gallery in Eleusis, using lathe. On this marble 
inscription is written among others “... a copper alloy from Marion (Cyprus) must be 
used, consisting of 11 parts copper and one part tin ...” This alloy is called bronze 
today. The parts of Antikythera Mechanism are made of bronze (see Sect. 3.2.1.1). 
Subsequently is written “... Turn the axes according to the example ...” This inscription 
shows that many years before the creation of the Mechanism, the Greeks had and used 
lathes. This is also apparent from other sources [25]. 

For the machining of bronze pieces, steel cutting tools are necessary. It follows from 
several sources that the Greeks at that time had such machine tools and cutting tools 
[25]. 
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Fig. 29. Marble inscription fourth century BC. Archaeological Museum of Eleusis. 
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Abstract. The quality of digital documentation of cultural heritage sites and 
objects is influenced significantly by the complexity of the subject. Complex- 
ity is an encompassing concept comprised of a range of factors including material 
properties and surface characteristics. Highly reflective items pose a challenge to 
traditional workflows, which has been previously addressed with a range of miti- 
gating techniques and strategies, often including cross-polarised photogrammetry 
lighting setups to omit specular type reflections. However, in the removal of reflec- 
tions to achieve more robust surface geometry, important information about the 
specular albedo of the surface can be lost. In this paper we demonstrate an accessi- 
ble single camera cross-polarised photogrammetry workflow to retain the diffuse 
and specular albedo information. The results have enabled qualitative assessments 
about the individual objects and their materials through the workflow. The items 
discussed include ceramic, metal and wood material types, and through the sep- 
aration of specular information offer unique improvements to their visualisation 
and insights to their physical condition for conservation purposes. 


1 Introduction 


There are many ways in the field of digital documentation that artefacts can add layers of 
complexity to their capture. Each artefact is unique and will therefore offer unique chal- 
lenges. One such challenge that spans a great many objects is that of shiny and reflective 
surfaces. Glossy finishes that return specular (or mirror-like) reflections negatively affect 
3D data capture techniques that rely on light, such as laser scanning and photogrammetry. 
The adverse impact on the quality of the data capture results in lower geometric accuracy 
and ‘bakes’ specular reflections into image textures. This in turn reduces the usefulness 
of the output models for cultural heritage purposes, where detailed condition monitoring 
and inspection are essential conservation objectives. This paper explores the use of a 
cross-polarised photogrammetry workflow for complex surfaces in a cultural heritage 
context, with a predominantly qualitative discussion of results with a conservation focus. 
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The items were digitally documented as part of Historic Environment Scotland’s (HES) 
Rae Project (Historic Environment Scotland 2020), a commitment to digitally docu- 
ment our 336 Properties in Care and the 40,000 + collections objects they house - many 
of which have ceramic glazes, polished metal varnishes, and other complex reflective 
or glossy surfaces. A general introduction to digital documentation and the Rae project, 
including some of the key technologies, guidelines, methodologies and HES case studies 
are outlined in the HES Short Guide 13: Applied Digital Documentation in the Historic 
Environment (Historic Environment Scotland 2018). 

The approach to the complexity of digitisation in this paper links with the objectives 
of the European Commission funded VIGIE-2020-654 study regarding the quality and 
complexity of 3D digitisation of tangible cultural heritage. The study identified factors 
governing both the complexity of the digital documentation process, and quality of the 
outputs. Figure | is a diagram produced by the VIGIE-2020-654 study showing some 
key parameters as relevant to this paper. Of these factors, material and condition are both 
central to the investigation and documentation of items with complex or highly reflective 
surfaces (Fig. 1A). For quality parameters in Fig. 1B, 3D geometry, image texture and 
material properties are the most important factors for assessing the quality of outputs. 
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Fig. 1. Diagrams from the VIGIE-2020-654 study (A) ‘Object’ segment of complexity (B) ‘Image 
texture’ segment as a factor of quality. 
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1.1 Background 


Previous work addressing this problem has shown that the use of a cross-polarised light 
approach for photogrammetry can improve the results of data capture for reflective 
objects and surfaces in this context (Menna et al. 2016; Noya et al. 2015; Nicolae et al. 
2014). Furthermore, the photogrammetric capture of materials that exhibit even more 
complex properties, such as subsurface scattering, has also been explored using this con- 
trolled methodology (Angheluta and Radvan, 2020). Cultural artefacts are constructed 
from a range of materials, and items or environments that have heterogeneous surfaces 
represent one of the greatest challenges both in terms of data capture and visualisation of 
the results. Metals paired with detailed painted surfaces are commonly encountered, and 
whilst both can be highly reflective, behave differently under a cross-polarised configura- 
tion (Hallot and Gil 2019). For physical records (‘reflective originals’), often documents, 
maps etc., the ISO standard 19264-1:2021 sets out key quality control parameters for 
imaging systems (International Organization for Standardization 2021). While it does 
not address the use of polarised light explicitly, it makes clear that the use of imaging 
systems is specific to different use-cases where different outputs may be required. 

Previous approaches use similar photographic equipment and cross-polarised con- 
figurations to achieve their results, with the primary objective of improving both the 
geometric accuracy of the 3D model, as well as the diffuse albedo texture map (includ- 
ing colour accuracy). This is essential to improve the quality of the data capture processes 
and outputs. However, to improve visualisation of most of these complex items and mate- 
rials requires the isolation of both diffuse albedo and specular albedo to more closely 
represent the combined reflectance characteristics through the Bidirectional Reflectance 
Distribution Function (BRDF) (Bhandari et al. 2022 p. 313). Developments in the field 
of computer graphics provide examples of cross-polarised photographic systems and 
workflows to separate these reflection maps and improve visualisation of complex and 
spatially varied surfaces, including static objects, and more complex subjects such as 
human faces (Alexander et al. 2009; Ghosh et al. 2010). 


1.2 Our Approach 


As shown in this paper, we have bridged these approaches by adapting a relatively 
common single camera cross-polarised photogrammetry workflow to isolate the specular 
albedo map from the diffuse albedo map. This has the dual benefit of improvements to 
the data capture and visualisation of these cultural artefacts, some of which exhibit 
complex heterogeneous surfaces. For many items, the specular reflection characteristics 
can contain significant information about the condition or material of the artefacts, such 
as the presence and quality of ceramic glazing, metal oxidation or tarnishing and polished 
areas. This information directly informs subsequent inspection and condition monitoring 
activities, which can then inform decision making for conservation strategies. 

Working in partnership with Spectrum Heritage (a commercial digital documentation 
team), HES explored a cross-polarised photogrammetry workflow that would enable 
the removal of specular reflections from the model diffuse albedo map and facilitate 
the calculation of a map approximating specular albedo to improve visualisation. The 
subjects of this workflow are collections items within the care of HES, selected for their 
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varied materials and relative complexity of geometry. Figure 2 illustrates an example 
of one of the digitally documented items, with a visual breakdown of the maps used to 
produce the image rendered using a physically based rendering shader. 


HISTORIC ARAINNEACHD 
ENVIRONMENT | EACHDRAIDHEIL 


SCOTLAND ALBA 


Fig. 2. Composite image showing different workflow outputs overlaid. From left to right: Final 
render; specular albedo; diffuse albedo; optimised geometry wireframe. Not pictured: tangent 
space normal map. Item is Greyhound Desk Weight, accession number DC351. 


2 Methodology 


“Cross-polarised’ photography is a workflow used to achieve reflectionless photography 
using ‘polarised’ light (Bhandari et al. 2022). Some light is already strongly polarised, 
including natural sky light as the sun’s light scatters in the sky (Hanlon 2018). By util- 
ising polarising filters in front of the lens to strip out this polarised light, photographers 
can remove strong sky reflections from certain surfaces, like water or glass. In a con- 
trolled lighting environment such as a studio, or otherwise under artificial lighting, these 
unpolarised light sources must first be polarised to enable the separation of diffuse and 
specular reflections. 

When polarised light bounces off a rough surface that diffusely reflects light, much of 
that light will become unpolarised. However, smooth microstructure surfaces can reflect 
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light (often referred to as ‘specular’ reflections) in a way that maintains the light’s polar- 
isation, such as glass, glazed ceramics, and many plastics (Dorsey et al. 2008, p. 62). It is 
worth noting that polarised light, and cross-polarised inspection methodologies already 
see use in the field of conservation with microscopy to assist with the identification of 
mineralogical composition in historic mortars and archaeological ceramics (Riederer 
2004; Blaeuer and Kueng 2007). 

The key aspects of the methodology can be broken down into two parts: 1) data 
capture and 2) data processing. 


2.1 Data Capture 


The setup is not dissimilar to a typical photogrammetric approach to digitally document- 
ing artefacts. Figure 3 illustrates this set up and team at Spectrum Heritage capturing 
images of a collections item in HES’s care. The hardware utilised included a remotely 
triggered DSLR camera controlled by a laptop, relatively diffused two-point lighting, 
and a turntable to place the item on. In addition, colour calibration charts were used 
with metric distance scales to ensure the final product was colour-accurate and correctly 
scaled. 


Fig. 3. Physical photographic setup enabling remote triggering and image quality control in real- 
time to ensure optimum removal of polarised light. Pictured: Clara Molina Sánchez and David 
Vacas-Madrid. 
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The cross-polarised workflow was achieved using 3D printed bespoke filters to hold 
linearly polarised film, which were rotated to dial in the angle of the incident light. Linear 
polariser film was placed over two LED flash lights, and a polariser filter was also fixed 
on the camera lens. Then, the polariser filter was rotated until it was perpendicular to 
the polariser film, therefore achieving a “cross-polarisation”. In order to calibrate the 
polarity of the incident light, a glossy black ball was initially placed within the frame, 
allowing the team to align the two light sources relative to the lens filter. 

This capture process was time extremely intensive, each object taking around a day 
to capture properly. Every photograph was taken twice: one photograph with the full 
specular reflections, and its twin with reflections removed by rotating the filter 90 degrees 
(non-polarised and polarised respectively), as highlighed in Fig. 4. 


2.2 Data Processing 


Reality Capture 

In the office, the photographs were processed into 3D models using photogrammetric 
software Reality Capture (2021). Both the polarised and non-polarised images were 
imported and aligned in one project, whilst only using the polarised images to create 
the mesh to reduce the influence of the specular reflections on geometric reconstruction. 
Several levels of detail of model were produced in line with the requirements of the 
project, with the highest level of details enabling possible outputs such as 3D repro- 
duction, and to facilitate the baking of high-resolution details to lower resolution model 
iterations. Optimised models were produced using geometric ‘simplification’ to reduce 
the triangle/vertex count, whilst ensuring manifold geometry and UV mapping to enable 
the baking of tangent-space normal maps. 

To texture the mesh, both the non-polarised and polarised images were successively 
used to create two diffuse texture maps with identical alignment. For the final diffuse 
texture, the team used the albedo map without reflections. The team then calculated 
the ‘reflection’ map by overlaying and subtracting the resulting two textures, described 
further in the following section. 


Texture Maps 

The application of different texture maps is what brings the 3D representation of artefacts 
as close to reality as possible. Physically based rendering (PBR) refers to this idea of using 
realistic shading and lighting models alongside measured surface values to represent real- 
world materials as accurately as possible. This rendering approach is currently used by 
most modern 3D software and 3D web viewers, such as Sketchfab (2021). 

As mentioned above, the team made the specular or ‘reflection’ map from the 
polarised and cross-polarised diffuse maps by calculating the difference between the 
two using Substance 3D Designer (2021) and Adobe Photoshop. Figure 5 illustrates the 
diffuse and specular albedo maps alongside the rendered resulting 3D model for the 
Wassail Bowl from the Duff House collection. 

As material shaders on different platforms handle input values uniquely, the differ- 
ence maps often needed further processing in the image editor to tailor them for the final 
output. The workflow below illustrates these stages. 
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Fig. 4. The photogrammetry set up (top) including the two flash lights, and camera with polarised 
filter in order to achieve first full specular reflection, or non-polarised (bottom left), and one 
polarised image without reflection (bottom right). Item is Flower Holder, accession number 
DC475. 


Specular 

This map contains the specular albedo reflectance information from surfaces. In 
order to allow the renderer to convey the model’s spatially varying reflectance values, 
black areas are shown as completely matt, while white areas are completely reflective. 

To achieve this, the team overlaid the two diffuse albedo textures derived from the 
polarised and non-polarised images and calculated the difference and converted that 
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Fig. 5. Image illustrating the different texture maps applied to an artefact. From left to right: 
Diffuse albedo; specular albedo; final render. Not pictured: tangent space normal map. Item is 
Wassail Bowl, accession number DC334. 


to greyscale in Substance 3D Designer (Fig. 6). This greyscale map produced through 
that image-difference process represents the intensity of specular reflections (specular 
albedo) on an object’s surface. 

In order to ensure the reflection output will be rendered correctly when used in 
the material shader, this raw ‘reflection’ map will need further image processing. The 
extents of this image processing will vary depending on a range of factors, including 
but not limited to the initial capture conditions, the material shader, and the final output 
platform. 

Roughness/Glossiness 

These maps describe the microsurface of an object and dictate the sharpness of any 
reflections. White areas represent roughness, and black indicates smoothness. Different 
platforms, rendering systems and material shaders vary in their interpretation of these 
values. For this project, as Sketchfab ( ) was one of the intended dissemination plat- 
forms for the optimised models, the levels were adjusted by normalising the histogram 
to evenly represent the full range of the grayscale values. This obtains a greater contrast, 
helping the web viewer to recognise the roughness of the surface more effectively. 

Metallic 

Generated when there are instances of spatially varied metallic areas on an object, 
this map indicates to the shader aspects of the object that are metal. Metallic maps may 
be represented either in greyscale or colour across different rendering systems. For this 
project the maps were produced in grayscale, with white depicting areas of high metallic 
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Fig. 6. Illustration of Specular map creation in Substance 3D Designer. Left to right: Diffuse Map 
with cross-polarisation and without; the calculated difference; converted to greyscale; inverted; 
levels adjusted for final product. 


albedo, and non-metal/dielectric areas, denoted with black. There can be transitional grey 
values that indicate something covering the raw metal, such as dirt or tarnish. 

Normal and Ambient Occlusion 

Finally, tangent space surface normals and ambient occlusion (AO) maps are inte- 
gral to completing the models for visualisation. The baked surface normals enable the 
geometrically optimised model to render with finer details found in the surface topology. 
The AO map simulates the soft shadows that occur in the crevices of an object, adding 
an optional depth to the re-topologised model. 


3 Results and Discussion 


This section will explore the results for selected objects from the study by material 
type. It was assumed by the authors at the outset that different materials would respond 
differently throughout this workflow to the effect of cross-polarised light and produce 
varied results. The following qualitative comparison between objects and the results of 
the workflow will reflect on whether this technique improves the level of information 
captured and enhances inspection. This includes the clarity of visual features apparent 
in the diffuse and specular albedo image texture maps, and whether the approach better 
facilitates inspection of the object surface properties relating to its condition from a 
conservation perspective. 

The items are fully listed in the Appendix, with images showing input photographs 
alongside rendered outputs for visual comparison. 


3.1 Ceramic 


Cultural artefacts made of, or including, ceramic material makes up 3% of HES’ collec- 
tions. Ceramics often represent a challenging and mixed material to digitally document 
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with photogrammetry, due primarily to the variation of surface finishes that can range 
from completely matt (such as an unglazed earthenware vessel) through to a buff pol- 
ished or highly reflective glazed finish. Often, these surfaces may contain imperfections 
or characteristics created during the manufacture or subsequent use that can reveal impor- 
tant information about the item’s usage or condition. Ceramics are considered dielectric 
materials, though some may contain metallic/conductive components which may require 
the 3D model to use separate ‘metallic’ texture maps to render these areas appropriately 
(Westin et al. 2004). 


St Andrews Glazed Tile (SAC422) 

Item description: “This unusually decorated medieval floor tile is believed to be from 
Blackfriars church in St Andrews, the remains of which survive just off of South Street in 
town. Its Low Countries redware fabric indicates a trade connection. It may have been 
imported from Low Countries such as the Netherlands or Belgium’. Item dimensions: 
67 x 50 x 25 mm (HES, Vernon Collections Management System, “Object - SAC422’ 
accessed 25/10/21). 


Fig. 7. St Andrews Glazed Tile: Detailed area excerpt of UV texture sheet maps in the following 
order left-to-right: (A) diffuse albedo; (B) tangent-space surface normal; (C) specular albedo. 
Assession number SAC422. 


Discussion and Interpretation 

This ceramic tile includes a single glazed face (visible in Fig. 7), with an otherwise 
exposed fabric on all other sides. The fabric is a light red with dark grey inclusions, a 
rough matt surface and shows accreted remains of a light-coloured mortar. Crazing within 
the glaze was observed in both the cross-polarised and unpolarised texture maps. The 
cross-polarised results show the differences between the diffuse and specular albedo 
maps, the former of which is clear and does not suffer from the “baked in’ veiling 
reflection often visible in image textures for highly reflective objects. In the specular 
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albedo map, the glazed surface shows detailed and dense surface scratches, gouges, 
marks, and damage to the glaze and underlying decoration revealing areas of fabric. 
This pattern of damage is consistent in distribution across the glazed area, and the linear 
scratches mostly appear random in direction and length, with a few strong exceptions 
that are parallel and may be attributable to the same source. 


Japanese Kutani Vase (TRH135) 

Item description: “This is one of a pair of similar but not identical Japanese Kutani 
vases at Trinity House. The vases are painted in reds, blues, and gilts, with panels of 
flower spray. Kutani ware is a style of Japanese porcelain traditionally from Kutani, 
now a part of Kaga, Ishikawa, in the former Kaga Province.” Item dimensions: 690 x 
310 mm (HES, Vernon Collections Management System, ‘Object - TRH135’ accessed 
25/10/21). 


Fig. 8. Japanese Kutani Vase - Detailed area excerpt of UV texture sheet maps in the following 
order left-to-right: (A) diffuse albedo; (B) tangent-space surface normal; (C) specular albedo. 
Assession number TRH135. 


Discussion and Interpretation 

With multiple painted, gilt and glazed surfaces, the finish on this vase presents a 
challenge to capture photogrammetrically. It is also an opportunity to more accurately 
visualise the mixed materials of the surface decoration. Figure 8(B) shows the relatively 
smooth surface of the normal map for the body of the vase. Figure 8(C) immediately 
shows the higher specular albedo associated with the gilt areas, and the relatively high 
average specular value across the body of the vase should be noted due to the reflective 
glaze. An unpolarised photogrammetry approach would not differentiate between the 
gilt areas and those with similar albedo levels or colour values. 

As seen in Fig. 9, under normal ‘unpolarised’ lighting conditions the painted and 
gilt areas are occluded where the specular highlight falls on the vase body, with a light 
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veiling sheen across the surrounding area. The combination of these effects reduces the 
contrast of the diffuse albedo map, obfuscating detail. 


Fig. 9. An excerpt of a cropped photograph from the ‘unpolarised’ photogrammetry imagery for 
item TRH135. Note the specular highlight from the studio light visible in the top centre area. 


The specular albedo map in Fig. 8(C) enables spatially varied specular reflection 
through the material shader. However, the metallic map is used in addition to denote the 
spatially varied metallic response from the object surface. The difference map produced 
from the unpolarised and cross-polarised diffuse albedo maps is compared to the unpo- 
larised diffuse map in Fig. 10. Figure 10(B) shows the metallic gilt areas, with a strong 
gold response highlighting the areas on the vase containing decorative gilt finish, in con- 
trast to the painted and unpainted areas. The influence of the glaze in reducing contrast 
through veiling light is negligible. For material shaders that require a grayscale raster 
image for the metallic map, such as Sketchfab’s PBR shader (2021), this ‘difference 
map’ can be further processed as discussed in Sect. 2.2. 


3.2 Metal 


Metallic items, or artefacts and surfaces with metal components, can exhibit a range of 
reflective properties largely dependent on the condition and finish of the surface. For 
example, heavily oxidised metal surfaces may show lower specular albedo values, such 
as rusted iron or a bronze verdigris are present. Highly polished metals, often associ- 
ated with jewellery can reach a mirror-like reflective level. Brushed metal finishes can 
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Fig. 10. Comparison between (A) the unpolarised diffuse albedo map, and (B) the difference map, 
calculated from unpolarised vs. cross-polarised diffuse maps. Minor histogram levels adjustments 
were made to increase contrast for the higher intensity values. 


often exhibit anisotropic reflections, which can be represented by more complex phys- 
ically based material shaders. As conductors, bare (i.e., unpainted, un-oxidised) metals 
behave differently to dielectric materials in physically based rendering systems (Westin 
et al. 2004). Usually these have a dedicated parameter for material shaders representing 
spatially homogenous surfaces, or a texture map for spatially varied surfaces. 


Greyhound Desk Weight (DC351) 
Item description: “This is one of a pair of greyhound desk weights, dating to the late 
18th or early 19th-Century. Although very French in their making, cast gilt-finished 
metal (ormolu) and ornament of laurel-leaf swags, these little dogs have had a long 
classical tradition. An example of a similar sculpture are the ‘Townley Greyhounds’, 
which date from the 2nd century BC and were an inspiration to a generation of artist 
animaliers (animal painters) in the 19th century.” Item dimensions: w172 mm (HES, 
Vernon Collections Management System, ‘Object - DC351’ accessed 25/10/21). 

Discussion and Interpretation 

The cross-polarised results for the metal Greyhound Desk Weight show a striking 
difference to the other items. The metallic surfaces are finished to a high sheen, which 
shows in the diffuse albedo when compared side by side with the cross-polarised and 
unpolarised workflows (as visible in Fig. 12). The specular albedo contains high values, 
with surface scratches, dents and marks highly visible. 

The metallic composition and condition of item DC351 has resulted in a surface of 
varied colour. This can be seen on the body of the Greyhound figure, where a visible 
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Fig. 11. Greyhound Desk Weight - Detailed area excerpt of UV texture sheet maps in the following 


order left-to-right: (A) diffuse albedo; (B) tangent-space surface normal; (C) specular albedo. 
Assession number DC351. 


gradient appears to show a transition from lighter bronze/gold colour to a deeper red. 

This can be seen partly in Fig. 11(A), and more closely in Fig. 12. These differences in 

colour between swatches A and B appear visible to the naked eye between the techniques. 

To better define the differences between the unpolarised and cross-polarised approaches 

we have opted to use the CIEDE2000 method (Sharma et al. 2005) to illustrate: 
Unpolarised workflow CIEDE2000 


L a b 
Swatch A 26 
Swatch B 22. 10 
AE 6.3843 

Cross-Polarised workflow CIEDE2000 

L b 
Swatch A 17 10 
Swatch B 13 9 
AE 7.4982 


These results indicate that the CIEDE2000 shows greater difference between the 
swatches from the cross-polarised workflow. The extent of the colour difference is in the 
perceptible range (>1) to the observer, which suggests an incremental improvement in 
the ability to distinguish between the colour of the surface characteristics of the metal 
item using the cross-polarised workflow. 
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It may be that this variation in colour is due to the deterioration or change to the 
gilt-finished ormolu surface. If this is the case, this comparison illustrates the ability 
to better differentiate between the areas where the finish is intact, document its present 
condition and better record any further changes. 


Fig. 12. Comparison between unpolarised (left) and cross-polarised (right) diffuse albedo texture 
maps for item DC351. Swatches A and B (for both images) are sampled from identical locations 
and represent averaged colour values for identical areas on both maps. 


A further observation is the presence of fine particulate matter on the item surface, 
e.g., dust, lint, etc. Typically, this matter can collect on an item depending on storage, or 
more likely, display conditions. As shown in Fig. 13, the presence of this is immediately 
obvious in the cross-polarised image texture and the derived specular map. This is due to 
the highly diffuse characteristics of the particulate matter. This is nearly indistinguishable 
in the unpolarised (i.e., typical) approach due to the occluding glare of the reflective 
surface from the lighting conditions. The ability to detect this could be valuable for 
fine-level assessment of condition, for example, of items before and after public display 
(where it may be exposed to uncontrolled environmental conditions), or before and after 
the item returns from a loan. 


3.3. Wood 


As a natural, organic material, wood can vary hugely in its visual characteristics. Its use 
for cultural artefacts is usually accompanied by different decorative and/or protective 
finishes that can include the use of pigments in paint application, varnishes and waxes, 
that can all alter its reflective properties. Fundamental differences such as softwoods 
versus hardwoods can govern many aspects of the visual appearance. Whilst items made 
from bare or natural wood might have a visually consistent presentation for the wood 
substrate, as an assembly an item might include metallic fixings or other inorganic 
materials. Like ceramics, wood is a dielectric material. Cross-polarised light has also 
been used for specific research applications into wood related to enhancing tree-ring 
inspection (Gartner et al. 2015). 
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Fig. 13. Comparison between excerpts of item DC351 image texture maps showing diffuse albedo 
(unpolarised, left), diffuse albedo (cross-polarised, centre) and specular albedo (right). 


Stirling Head (STC022) 

Item description: “Oak roundel, part of a series of carvings that decorated the ceiling 
of the King’s Presence Chamber at Stirling Castle. The pose and costume in this Head 
appears in contemporary paintings of flamboyant noblemen. He is shown wearing an 
elaborate and stylish quilted doublet. The Stirling Heads are constructed of 3 panels, 
originally only secured by glue. The great majority of the Heads, however, are in part 
constructed in 2 ply, an additional board or block having been placed over the middle 
board to enable the carver to model the central portion of the medallion in high relief. 
Nail holes are apparent in the borders which indicate the way in which the Heads were 
fixed to the ceiling.” Item dimensions: diameter 762mm. (HES, Vernon Collections 
Management System, “Object - STC022’ accessed 25/10/21). 

Discussion and Interpretation 

Initially the specular map results are apparent across the entire surface of the Oak 
roundel. A subtle, consistent level of reflectance can be observed from the specular 
map in Fig. 14(C). Areas of obvious occlusion and a reduction in specular response 
from surface geometry can be found for carved recessed details, such as the figure’s 
mouth, fingers and details of the clothing. This can be compared with the normal map 
in Fig. 14(B), showing a geometric correlation to these details. 

Figure 15 compares an unpolarised photograph with a cross-polarised photograph, 
showing the veiling effect of the specular response for the underlying wood tone. Whilst 
the specular response does not appear to affect the legibility of wood grain, it does 
occlude the surface colour of the wood, particularly around the specular highlight in the 
image centre. 
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Fig. 14. Stirling Head - Detailed area excerpt of UV texture sheet maps in the following order 


left-to-right: (A) Diffuse albedo; (B) tangent-space surface normal; (C) specular albedo. Assession 
number STC022. 


Fig. 15. Split image showing side-by-side results for photographs of item STC022 from (A) the 
cross-polarised setup (top half) and (B) unpolarised photograph (bottom half). 
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The ‘figure’ of wood (often simplified, or referred to, as “grain’) is a key feature of the 
material’s appearance. It can provide key diagnostic information for inspection purposes, 
both in terms of identifying wood species and the orientation of the surface from cutting 
(Hoadley 2000, p. 25). Figure 16 illustrates an example wherein the characteristics of 
this figure can be seen. Initially there appear to be few differences between the cross- 
polarised and unpolarised results, however the specular albedo map and surface normal 
map reveal that the undulations of the surface geometry caused by the carved wood 
surface interacting with the figure of the wood to affect the reflectance characteristics. 
The specular map clearly captures the recesses and raised areas in detail, allowing more 
accurate visualisation and interrogation of the surfaces in the captured data. 


Fig. 16. Comparison of item STC022 between excerpts from the diffuse albedo (cross-polarised, 
left), diffuse albedo (unpolarised, centre-left), specular albedo (difference product, centre-right) 
and tangent-space surface normal (right). 


It was unclear from the outset of the project whether there would be an appreciable 
benefit from using the cross-polarised workflow for a wooden item. The possible benefits 
demonstrated above should be weighed against the increased fieldwork/data capture and 
processing time. This is particularly true for sensitive organic items that may normally 
be stored in a protective environment. 


4 Conclusion 


Structure from Motion photogrammetry has become widely embraced as a powerful and 
flexible 3D data capture technique. However, as indicated in this paper more complex 
materials, surfaces and items can hinder robust and consistent results. The technique has 
seen development within the realm of computer imaging and rendering, which has driven 
more sophisticated modes of data capture and visualisation. However, best practices and 
guidance for geospatial application of photogrammetry is still primarily focussed on 
recording ‘diffuse’ surfaces within a measurable survey tolerance. This often neglects 
the rich and spatially varied material characteristics seen across cultural heritage that 
are able to be represented through sophisticated material shaders. 
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While not unlike a typical photogrammetry workflow, this cross-polarised method 
of capture provides information about cultural heritage artefacts that another technique 
could miss. Indeed, in some cases imperfections on the surface that are only revealed 
with these methods change the appearance of an object entirely. 

More than improving the visualisation of artefacts, there are potential conservation 
applications in using richer material shaders and a greater depth of surface information, 
including: 


e Capturing the condition of reflective surface finishes such as varnishes and glazes, 
for example, where crazing is visible 

e Recording and visualising reflective metallic surfaces and distinguishing them from 
oxidised, worn or patinated areas for condition assessment 
To investigate and identify areas of trace pigments or gilt finishes on ceramic surfaces 
Documenting ‘micro-structure’ surface scratches, which may otherwise not appear 
in standard capture 

e The documentation of reflective collections items, with glossy varnishes or pol- 
ished metallic for example, where specular reflections negatively influence the 
reconstruction of accurate 3D surface geometry 

e Asaquality control process for comparative condition checks on artefacts loaned from 
the collections to establish condition before and after, which can inform subsequent 
treatment approach 


The workflow described here is far more involved that the standard approach to 
single camera photogrammetry, extending the time required during both data capture and 
processing stages. That said, these results are promising when applied to documenting 
complex objects that are a challenge to capture using unpolarised light workflows. The 
improvements to the quality of image textures and the 3D data captured for these objects 
has been identified and explored for these cultural heritage artefacts. These items are 
exemplars of complex surfaces with varied materials, and spatially varied reflective 
properties. The work described here has established a foundation for further research in 
creating a standardised cross-polarised workflow in a heritage context in the future. 


Appendix 
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Item 
description 
and 
accession 
number 


Input image from 


dataset 


Sexfoil Cup, 
Arbroath 
Abbey 


ARB069 


Wassail Bowl, 
Duff House 


DC334 


Desk Weight, 
Duff House 


DC351 


Rendered image 


and lighting setup 


configured 


non-polarised photography | with basic PBR material shader 
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Flower Hold- 
er, 
Duff House 


DC475 


Tureen, 
Duff House 


DC490 


Baluster Vase, 
Duff House 


DC544 


Recorder, 
Fort George 
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Glazed Tile, 
St Andrews 
Cathedral 


SAC422 


Stirling Head, 
Stirling Castle 


STC022 


Kutani Vase, 
Trinity House 


TRH135 
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Whale 
Eardrum, 
Trinity House 


TRH1908 


Axe Head, 
Whithorn 
Priory 


WHP.BO7 
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